2024-10-24 16:16:35,281 - DEBUG - [10.0.0.27]: Running cmd : hostname 2024-10-24 16:16:35,480 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1 2024-10-24 16:16:35,480 - DEBUG - [10.0.0.27]: Running cmd : hostname -f 2024-10-24 16:16:35,554 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1. 2024-10-24 16:16:35,554 - DEBUG - [10.0.0.27]: Running cmd : docker ps -a 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2024-10-24 16:16:35,684 - DEBUG - Output : NAMES contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_kafka_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 webui_web_1 webui_job_1 config_devicemgr_1 config_schema_1 config_nodemgr_1 config_dnsmasq_1 config_api_1 config_svcmonitor_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:16:35,685 - DEBUG - [10.0.0.27]: Running cmd : getent hosts 10.0.0.27 | head -n 1 | awk '{print $2}' 2024-10-24 16:16:35,758 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1. 2024-10-24 16:16:35,759 - DEBUG - [10.0.0.27]: Running cmd : ip addr show | grep 'inet .*/.* brd ' | awk '{print $2}' 2024-10-24 16:16:35,814 - DEBUG - Output : 10.0.0.27/24 172.17.0.1/16 10.20.0.10/24 2024-10-24 16:16:35,815 - DEBUG - [10.0.0.27]: Running cmd : ip addr show dev vhost0 | grep 'inet .*/.* brd ' | awk '{print $2}' 2024-10-24 16:16:35,889 - DEBUG - Output : 10.20.0.10/24 2024-10-24 16:16:35,890 - DEBUG - [10.0.0.27]: Running cmd : getent hosts 10.20.0.10 | head -n 1 | awk '{print $2}' 2024-10-24 16:16:35,957 - DEBUG - Output : 2024-10-24 16:16:37,804 - INFO - Using existing project ['default-domain', 'admin'](a6890942-6a5a-4756-95e0-8a3f393aa814) 2024-10-24 16:16:37,997 - INFO - ================================================================================ 2024-10-24 16:16:37,998 - INFO - STARTING TEST : test_contrail_status 2024-10-24 16:16:37,999 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2024-10-24 16:16:38,144 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:16:38,152 - DEBUG - Nothing to compare xmpp stats {'10.0.0.27': {'10.20.0.10': '0'}} with 2024-10-24 16:16:38,152 - INFO - Initial checks done. Running the testcase now 2024-10-24 16:16:38,152 - INFO - 2024-10-24 16:16:38,153 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:16:38,276 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:16:38,276 - INFO - 10.0.0.27 2024-10-24 16:16:38,276 - INFO - config 2024-10-24 16:16:38,284 - INFO - config-nodemgr:active 2024-10-24 16:16:38,290 - INFO - api-server:active 2024-10-24 16:16:38,296 - INFO - schema:active 2024-10-24 16:16:38,296 - INFO - svc-monitor:inactive 2024-10-24 16:16:38,296 - INFO - device-manager:inactive 2024-10-24 16:16:38,296 - INFO - config-database 2024-10-24 16:16:38,296 - INFO - config-cassandra:active 2024-10-24 16:16:38,296 - INFO - config-zookeeper:active 2024-10-24 16:16:38,296 - INFO - config-rabbitmq:active 2024-10-24 16:16:38,297 - INFO - vrouter 2024-10-24 16:16:38,303 - INFO - vrouter-nodemgr:active 2024-10-24 16:16:38,309 - INFO - agent:active 2024-10-24 16:16:38,310 - INFO - control 2024-10-24 16:16:38,316 - INFO - control-nodemgr:active 2024-10-24 16:16:38,323 - INFO - control:active 2024-10-24 16:16:38,323 - INFO - named:active 2024-10-24 16:16:38,331 - INFO - dns:active 2024-10-24 16:16:38,331 - INFO - analytics 2024-10-24 16:16:38,338 - INFO - analytics-nodemgr:active 2024-10-24 16:16:38,344 - INFO - analytics-api:active 2024-10-24 16:16:38,351 - INFO - collector:active 2024-10-24 16:16:38,351 - INFO - analytics-database 2024-10-24 16:16:38,351 - INFO - analytics-cassandra:active 2024-10-24 16:16:38,358 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:16:38,366 - INFO - query-engine:active 2024-10-24 16:16:38,366 - INFO - webui 2024-10-24 16:16:38,367 - INFO - webui:active 2024-10-24 16:16:38,367 - INFO - webui-middleware:active 2024-10-24 16:16:38,367 - INFO - redis:active 2024-10-24 16:16:38,367 - INFO - analytics_snmp 2024-10-24 16:16:38,372 - INFO - snmp-collector:active 2024-10-24 16:16:38,373 - INFO - snmp-topology:active 2024-10-24 16:16:38,373 - INFO - analytics_alarm 2024-10-24 16:16:38,373 - INFO - alarmgen:active 2024-10-24 16:16:38,373 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:16:38,373 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0 2024-10-24 16:16:48,373 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:16:48,473 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:16:48,473 - INFO - 10.0.0.27 2024-10-24 16:16:48,473 - INFO - config 2024-10-24 16:16:48,480 - INFO - config-nodemgr:active 2024-10-24 16:16:48,486 - INFO - api-server:active 2024-10-24 16:16:48,491 - INFO - schema:active 2024-10-24 16:16:48,491 - INFO - svc-monitor:inactive 2024-10-24 16:16:48,492 - INFO - device-manager:inactive 2024-10-24 16:16:48,492 - INFO - config-database 2024-10-24 16:16:48,492 - INFO - config-cassandra:active 2024-10-24 16:16:48,492 - INFO - config-zookeeper:active 2024-10-24 16:16:48,492 - INFO - config-rabbitmq:active 2024-10-24 16:16:48,492 - INFO - vrouter 2024-10-24 16:16:48,498 - INFO - vrouter-nodemgr:active 2024-10-24 16:16:48,504 - INFO - agent:active 2024-10-24 16:16:48,504 - INFO - control 2024-10-24 16:16:48,514 - INFO - control-nodemgr:active 2024-10-24 16:16:48,521 - INFO - control:active 2024-10-24 16:16:48,521 - INFO - named:active 2024-10-24 16:16:48,530 - INFO - dns:active 2024-10-24 16:16:48,530 - INFO - analytics 2024-10-24 16:16:48,537 - INFO - analytics-nodemgr:active 2024-10-24 16:16:48,544 - INFO - analytics-api:active 2024-10-24 16:16:48,552 - INFO - collector:active 2024-10-24 16:16:48,552 - INFO - analytics-database 2024-10-24 16:16:48,553 - INFO - analytics-cassandra:active 2024-10-24 16:16:48,559 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:16:48,569 - INFO - query-engine:active 2024-10-24 16:16:48,569 - INFO - webui 2024-10-24 16:16:48,569 - INFO - webui:active 2024-10-24 16:16:48,569 - INFO - webui-middleware:active 2024-10-24 16:16:48,569 - INFO - redis:active 2024-10-24 16:16:48,569 - INFO - analytics_snmp 2024-10-24 16:16:48,574 - INFO - snmp-collector:active 2024-10-24 16:16:48,575 - INFO - snmp-topology:active 2024-10-24 16:16:48,575 - INFO - analytics_alarm 2024-10-24 16:16:48,575 - INFO - alarmgen:active 2024-10-24 16:16:48,575 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:16:48,575 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 1 2024-10-24 16:16:58,576 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:16:58,652 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:16:58,652 - INFO - 10.0.0.27 2024-10-24 16:16:58,652 - INFO - config 2024-10-24 16:16:58,659 - INFO - config-nodemgr:active 2024-10-24 16:16:58,664 - INFO - api-server:active 2024-10-24 16:16:58,670 - INFO - schema:active 2024-10-24 16:16:58,670 - INFO - svc-monitor:inactive 2024-10-24 16:16:58,670 - INFO - device-manager:inactive 2024-10-24 16:16:58,670 - INFO - config-database 2024-10-24 16:16:58,670 - INFO - config-cassandra:active 2024-10-24 16:16:58,670 - INFO - config-zookeeper:active 2024-10-24 16:16:58,670 - INFO - config-rabbitmq:active 2024-10-24 16:16:58,671 - INFO - vrouter 2024-10-24 16:16:58,676 - INFO - vrouter-nodemgr:active 2024-10-24 16:16:58,682 - INFO - agent:active 2024-10-24 16:16:58,682 - INFO - control 2024-10-24 16:16:58,688 - INFO - control-nodemgr:active 2024-10-24 16:16:58,694 - INFO - control:active 2024-10-24 16:16:58,695 - INFO - named:active 2024-10-24 16:16:58,701 - INFO - dns:active 2024-10-24 16:16:58,701 - INFO - analytics 2024-10-24 16:16:58,707 - INFO - analytics-nodemgr:active 2024-10-24 16:16:58,712 - INFO - analytics-api:active 2024-10-24 16:16:58,718 - INFO - collector:active 2024-10-24 16:16:58,718 - INFO - analytics-database 2024-10-24 16:16:58,719 - INFO - analytics-cassandra:active 2024-10-24 16:16:58,725 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:16:58,734 - INFO - query-engine:active 2024-10-24 16:16:58,734 - INFO - webui 2024-10-24 16:16:58,734 - INFO - webui:active 2024-10-24 16:16:58,734 - INFO - webui-middleware:active 2024-10-24 16:16:58,734 - INFO - redis:active 2024-10-24 16:16:58,734 - INFO - analytics_snmp 2024-10-24 16:16:58,740 - INFO - snmp-collector:active 2024-10-24 16:16:58,740 - INFO - snmp-topology:active 2024-10-24 16:16:58,740 - INFO - analytics_alarm 2024-10-24 16:16:58,740 - INFO - alarmgen:active 2024-10-24 16:16:58,740 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:16:58,740 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 2 2024-10-24 16:17:08,741 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:17:08,833 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:17:08,833 - INFO - 10.0.0.27 2024-10-24 16:17:08,834 - INFO - config 2024-10-24 16:17:08,840 - INFO - config-nodemgr:active 2024-10-24 16:17:08,845 - INFO - api-server:active 2024-10-24 16:17:08,850 - INFO - schema:active 2024-10-24 16:17:08,851 - INFO - svc-monitor:inactive 2024-10-24 16:17:08,851 - INFO - device-manager:inactive 2024-10-24 16:17:08,851 - INFO - config-database 2024-10-24 16:17:08,852 - INFO - config-cassandra:active 2024-10-24 16:17:08,852 - INFO - config-zookeeper:active 2024-10-24 16:17:08,852 - INFO - config-rabbitmq:active 2024-10-24 16:17:08,852 - INFO - vrouter 2024-10-24 16:17:08,857 - INFO - vrouter-nodemgr:active 2024-10-24 16:17:08,863 - INFO - agent:active 2024-10-24 16:17:08,863 - INFO - control 2024-10-24 16:17:08,872 - INFO - control-nodemgr:active 2024-10-24 16:17:08,878 - INFO - control:active 2024-10-24 16:17:08,878 - INFO - named:active 2024-10-24 16:17:08,884 - INFO - dns:active 2024-10-24 16:17:08,884 - INFO - analytics 2024-10-24 16:17:08,890 - INFO - analytics-nodemgr:active 2024-10-24 16:17:08,896 - INFO - analytics-api:active 2024-10-24 16:17:08,902 - INFO - collector:active 2024-10-24 16:17:08,902 - INFO - analytics-database 2024-10-24 16:17:08,902 - INFO - analytics-cassandra:active 2024-10-24 16:17:08,908 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:17:08,914 - INFO - query-engine:active 2024-10-24 16:17:08,914 - INFO - webui 2024-10-24 16:17:08,915 - INFO - webui:active 2024-10-24 16:17:08,915 - INFO - webui-middleware:active 2024-10-24 16:17:08,915 - INFO - redis:active 2024-10-24 16:17:08,915 - INFO - analytics_snmp 2024-10-24 16:17:08,920 - INFO - snmp-collector:active 2024-10-24 16:17:08,920 - INFO - snmp-topology:active 2024-10-24 16:17:08,920 - INFO - analytics_alarm 2024-10-24 16:17:08,920 - INFO - alarmgen:active 2024-10-24 16:17:08,921 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:17:08,921 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 3 2024-10-24 16:17:18,922 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:17:19,010 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:17:19,010 - INFO - 10.0.0.27 2024-10-24 16:17:19,010 - INFO - config 2024-10-24 16:17:19,018 - INFO - config-nodemgr:active 2024-10-24 16:17:19,024 - INFO - api-server:active 2024-10-24 16:17:19,031 - INFO - schema:active 2024-10-24 16:17:19,031 - INFO - svc-monitor:inactive 2024-10-24 16:17:19,031 - INFO - device-manager:inactive 2024-10-24 16:17:19,031 - INFO - config-database 2024-10-24 16:17:19,032 - INFO - config-cassandra:active 2024-10-24 16:17:19,032 - INFO - config-zookeeper:active 2024-10-24 16:17:19,032 - INFO - config-rabbitmq:active 2024-10-24 16:17:19,032 - INFO - vrouter 2024-10-24 16:17:19,038 - INFO - vrouter-nodemgr:active 2024-10-24 16:17:19,044 - INFO - agent:active 2024-10-24 16:17:19,044 - INFO - control 2024-10-24 16:17:19,050 - INFO - control-nodemgr:active 2024-10-24 16:17:19,061 - INFO - control:active 2024-10-24 16:17:19,061 - INFO - named:active 2024-10-24 16:17:19,068 - INFO - dns:active 2024-10-24 16:17:19,069 - INFO - analytics 2024-10-24 16:17:19,075 - INFO - analytics-nodemgr:active 2024-10-24 16:17:19,081 - INFO - analytics-api:active 2024-10-24 16:17:19,088 - INFO - collector:active 2024-10-24 16:17:19,088 - INFO - analytics-database 2024-10-24 16:17:19,088 - INFO - analytics-cassandra:active 2024-10-24 16:17:19,094 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:17:19,101 - INFO - query-engine:active 2024-10-24 16:17:19,101 - INFO - webui 2024-10-24 16:17:19,101 - INFO - webui:active 2024-10-24 16:17:19,101 - INFO - webui-middleware:active 2024-10-24 16:17:19,101 - INFO - redis:active 2024-10-24 16:17:19,101 - INFO - analytics_snmp 2024-10-24 16:17:19,106 - INFO - snmp-collector:active 2024-10-24 16:17:19,107 - INFO - snmp-topology:active 2024-10-24 16:17:19,107 - INFO - analytics_alarm 2024-10-24 16:17:19,107 - INFO - alarmgen:active 2024-10-24 16:17:19,107 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:17:19,107 - ERROR - Not all services up , Gave up! 2024-10-24 16:17:19,107 - INFO - Failed services are : defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:17:19,107 - ERROR - contrail-status failed 2024-10-24 16:17:19,233 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:17:19,240 - DEBUG - No XMPP flaps were noticed during the test 2024-10-24 16:17:19,240 - INFO - 2024-10-24 16:17:19,240 - INFO - END TEST : test_contrail_status : FAILED[0:00:42] 2024-10-24 16:17:19,240 - INFO - -------------------------------------------------------------------------------- 2024-10-24 16:17:21,892 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:17:24,547 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-80826256:ctest-vn22-42015148:ctest-vn22-42015148 2024-10-24 16:17:24,555 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-80826256:ctest-vn22-42015148:ctest-vn22-42015148 2024-10-24 16:17:24,563 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-80826256:ctest-vn22-42015148:ctest-vn22-42015148 2024-10-24 16:17:46,207 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-80826256:ctest-vn22-42015148 2024-10-24 16:17:46,214 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-80826256:ctest-vn22-42015148:ctest-vn22-42015148 2024-10-24 16:17:46,221 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-80826256:ctest-vn22-42015148:ctest-vn22-42015148 2024-10-24 16:17:46,388 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:17:50,574 - INFO - admin user already present 2024-10-24 16:17:51,548 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:17:53,986 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:17:53,992 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:17:53,998 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:17:55,133 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429:ctest-vn1-38876429 2024-10-24 16:17:55,140 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429:ctest-vn1-38876429 2024-10-24 16:17:55,148 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429:ctest-vn1-38876429 2024-10-24 16:18:04,409 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:b76b7ce2-a789-4900-b112-405e65059ce4 2024-10-24 16:18:04,417 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429:ctest-vn1-38876429 2024-10-24 16:18:04,423 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:b76b7ce2-a789-4900-b112-405e65059ce4 2024-10-24 16:18:15,745 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4d510427-3b0d-47c9-815b-97f2c915d068 2024-10-24 16:18:15,752 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:18:15,760 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4d510427-3b0d-47c9-815b-97f2c915d068 2024-10-24 16:18:17,371 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:b76b7ce2-a789-4900-b112-405e65059ce4 2024-10-24 16:18:17,379 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656 2024-10-24 16:18:17,387 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:18:17,395 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=3 2024-10-24 16:18:23,287 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429 2024-10-24 16:18:23,293 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429:ctest-vn1-38876429 2024-10-24 16:18:23,301 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-vn1-38876429:ctest-vn1-38876429 2024-10-24 16:18:23,621 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656 2024-10-24 16:18:23,628 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:18:23,635 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-50943791:ctest-fvn-74735656:ctest-fvn-74735656 2024-10-24 16:18:23,918 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:18:27,930 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:18:28,368 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:18:32,238 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:18:44,206 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:5462a85e-6d20-4e41-be75-ee185bf32337 2024-10-24 16:18:44,214 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestRoutersBasic-03450852:ctest-vn-75525297:ctest-vn-75525297 2024-10-24 16:18:44,223 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:5462a85e-6d20-4e41-be75-ee185bf32337 2024-10-24 16:18:50,316 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:26cf59de-33c8-487c-b3f6-a1a901aa3859 2024-10-24 16:18:50,325 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestRoutersBasic-03450852:ctest-ext_vn-43757685:ctest-ext_vn-43757685 2024-10-24 16:18:50,333 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:26cf59de-33c8-487c-b3f6-a1a901aa3859 2024-10-24 16:19:42,132 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:19:45,913 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:19:50,375 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-32874834:ctest-vn-32874834 2024-10-24 16:19:50,384 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-32874834:ctest-vn-32874834 2024-10-24 16:19:50,391 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-32874834:ctest-vn-32874834 2024-10-24 16:19:50,922 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-30745585:ctest-vn-30745585 2024-10-24 16:19:50,932 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-30745585:ctest-vn-30745585 2024-10-24 16:19:50,943 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-30745585:ctest-vn-30745585 2024-10-24 16:19:51,880 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-30745585 2024-10-24 16:19:51,889 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-30745585:ctest-vn-30745585 2024-10-24 16:19:51,900 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-30745585:ctest-vn-30745585 2024-10-24 16:19:52,257 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-32874834 2024-10-24 16:19:52,265 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-32874834:ctest-vn-32874834 2024-10-24 16:19:52,272 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-25596631:ctest-vn-32874834:ctest-vn-32874834 2024-10-24 16:19:52,539 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:19:56,327 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:20:00,140 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636:ctest-vn-95666636 2024-10-24 16:20:00,147 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636:ctest-vn-95666636 2024-10-24 16:20:00,155 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636:ctest-vn-95666636 2024-10-24 16:20:00,660 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461:ctest-vn-90440461 2024-10-24 16:20:00,669 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461:ctest-vn-90440461 2024-10-24 16:20:00,677 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461:ctest-vn-90440461 2024-10-24 16:20:09,067 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:0a4da5db-0e47-4719-aaed-d0e5edccf707 2024-10-24 16:20:09,074 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636:ctest-vn-95666636 2024-10-24 16:20:09,082 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:0a4da5db-0e47-4719-aaed-d0e5edccf707 2024-10-24 16:20:15,199 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66cc101a-8946-4cfb-861d-bc645639e85a 2024-10-24 16:20:15,209 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461:ctest-vn-90440461 2024-10-24 16:20:15,217 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66cc101a-8946-4cfb-861d-bc645639e85a 2024-10-24 16:20:27,278 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461 2024-10-24 16:20:27,285 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461:ctest-vn-90440461 2024-10-24 16:20:27,292 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-90440461:ctest-vn-90440461 2024-10-24 16:20:27,618 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636 2024-10-24 16:20:27,625 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636:ctest-vn-95666636 2024-10-24 16:20:27,631 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-30075056:ctest-vn-95666636:ctest-vn-95666636 2024-10-24 16:20:27,903 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:20:31,739 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:20:34,012 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:20:34,018 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:20:34,024 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:20:35,192 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:20:35,199 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:20:35,205 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:20:36,300 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:20:36,307 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:20:36,314 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:20:47,563 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:64b603da-b4d7-4639-bc17-81d3d98e2e88 2024-10-24 16:20:47,571 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:20:47,578 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:be7ca800-331d-48e0-9005-10515ff6ebcc 2024-10-24 16:20:47,585 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:20:47,591 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:2bd4a258-702e-40c4-b635-ee9a276a8007 2024-10-24 16:20:47,599 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:20:47,607 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:2bd4a258-702e-40c4-b635-ee9a276a8007 2024-10-24 16:20:47,616 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:64b603da-b4d7-4639-bc17-81d3d98e2e88 2024-10-24 16:20:47,623 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:be7ca800-331d-48e0-9005-10515ff6ebcc 2024-10-24 16:21:28,564 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9e9502e3-a565-482c-bb21-b9ffd90678a3 2024-10-24 16:21:28,572 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:21:28,579 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9e9502e3-a565-482c-bb21-b9ffd90678a3 2024-10-24 16:21:30,464 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:eed1082a-2cde-48a8-b08e-374546d23df2 2024-10-24 16:21:30,471 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:21:30,479 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:eed1082a-2cde-48a8-b08e-374546d23df2 2024-10-24 16:21:35,517 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:2bd4a258-702e-40c4-b635-ee9a276a8007 2024-10-24 16:21:35,525 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:64b603da-b4d7-4639-bc17-81d3d98e2e88 2024-10-24 16:21:35,534 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:be7ca800-331d-48e0-9005-10515ff6ebcc 2024-10-24 16:21:36,795 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926 2024-10-24 16:21:36,801 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:be7ca800-331d-48e0-9005-10515ff6ebcc 2024-10-24 16:21:36,808 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2024-10-24 16:21:36,822 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:21:36,999 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=2 2024-10-24 16:21:37,220 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Layer2RouteReq?x=2 2024-10-24 16:21:37,234 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926 2024-10-24 16:21:37,240 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:21:37,451 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=2 2024-10-24 16:21:37,675 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Layer2RouteReq?x=2 2024-10-24 16:21:37,695 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881 2024-10-24 16:21:37,705 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:64b603da-b4d7-4639-bc17-81d3d98e2e88 2024-10-24 16:21:37,716 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2024-10-24 16:21:37,728 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:21:37,899 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=4 2024-10-24 16:21:38,127 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Layer2RouteReq?x=4 2024-10-24 16:21:38,141 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881 2024-10-24 16:21:38,149 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:21:38,367 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=4 2024-10-24 16:21:38,600 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Layer2RouteReq?x=4 2024-10-24 16:21:38,611 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800 2024-10-24 16:21:38,620 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:2bd4a258-702e-40c4-b635-ee9a276a8007 2024-10-24 16:21:38,628 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2024-10-24 16:21:38,639 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:21:38,807 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=3 2024-10-24 16:21:39,036 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Layer2RouteReq?x=3 2024-10-24 16:21:39,054 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800 2024-10-24 16:21:39,062 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:21:39,271 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Inet4UcRouteReq?x=3 2024-10-24 16:21:39,496 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_Layer2RouteReq?x=3 2024-10-24 16:23:00,122 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800 2024-10-24 16:23:00,129 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:23:00,136 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-right_vn-69535800:ctest-right_vn-69535800 2024-10-24 16:23:00,467 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926 2024-10-24 16:23:00,474 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:23:00,482 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-left_vn-37146926:ctest-left_vn-37146926 2024-10-24 16:23:00,816 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881 2024-10-24 16:23:00,824 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:23:00,833 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-33645646:ctest-mgmt_vn-92359881:ctest-mgmt_vn-92359881 2024-10-24 16:23:01,112 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:23:04,990 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:23:14,598 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a11d348a-eb82-4397-8c18-c3859527ba85 2024-10-24 16:23:14,605 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestvDNSBasic0-40811773:ctest-vn1-vdns-44473942:ctest-vn1-vdns-44473942 2024-10-24 16:23:14,611 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a11d348a-eb82-4397-8c18-c3859527ba85 2024-10-24 16:23:26,972 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_DnsInfo? 2024-10-24 16:23:33,804 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:bd89f188-9331-4709-b252-9dc8544d6f79 2024-10-24 16:23:33,813 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestvDNSBasic0-40811773:ctest-vn1-vdns-44473942:ctest-vn1-vdns-44473942 2024-10-24 16:23:33,821 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:bd89f188-9331-4709-b252-9dc8544d6f79 2024-10-24 16:23:47,916 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_DnsInfo? 2024-10-24 16:23:59,271 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:24:03,020 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:24:12,711 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:10bda62a-4927-41a3-bd10-7b3dd3903f9e 2024-10-24 16:24:12,720 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVNx-32833409:ctest-vn-55379999:ctest-vn-55379999 2024-10-24 16:24:12,727 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:10bda62a-4927-41a3-bd10-7b3dd3903f9e 2024-10-24 16:24:18,823 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7a1f0185-558e-4682-bf98-6880cd827911 2024-10-24 16:24:18,831 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVNx-32833409:ctest-vn-55379999:ctest-vn-55379999 2024-10-24 16:24:18,837 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7a1f0185-558e-4682-bf98-6880cd827911 2024-10-24 16:24:33,153 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:24:37,011 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:24:44,586 - DEBUG - [10.0.0.27]: Running cmd : docker exec --privileged -it vrouter_vrouter-agent_1 /bin/bash -c 'echo "10.20.0.10 cn-jenkins-deploy-platform-ansible-os-2143-1-test" >> /etc/hosts' 2024-10-24 16:24:44,705 - DEBUG - Output : 2024-10-24 16:24:45,269 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dc4f5b9f-753f-422f-a15d-cb4b59dcc9d3 2024-10-24 16:24:45,276 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn2_metadata-39269717:ctest-vn2_metadata-39269717 2024-10-24 16:24:45,283 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dc4f5b9f-753f-422f-a15d-cb4b59dcc9d3 2024-10-24 16:24:57,839 - DEBUG - [10.0.0.27]: Running cmd : docker exec --privileged -it vrouter_vrouter-agent_1 /bin/bash -c 'sed -i '$ d' /etc/hosts' 2024-10-24 16:24:58,013 - DEBUG - Output : sed: -e expression #1, char 1: missing command 2024-10-24 16:25:00,655 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:25:00,966 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:25:06,629 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn-26506286:ctest-vn-26506286 2024-10-24 16:25:12,190 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a166b574-e65a-4e35-8b01-c3891b779e88 2024-10-24 16:25:12,202 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn-26506286:ctest-vn-26506286 2024-10-24 16:25:12,210 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a166b574-e65a-4e35-8b01-c3891b779e88 2024-10-24 16:25:18,345 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:245ab112-210f-4a25-b532-470a0f46380b 2024-10-24 16:25:18,352 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn-26506286:ctest-vn-26506286 2024-10-24 16:25:18,360 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:245ab112-210f-4a25-b532-470a0f46380b 2024-10-24 16:25:20,411 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:95d97106-f45d-48a8-8f04-227b79859a8c 2024-10-24 16:25:20,421 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn-26506286:ctest-vn-26506286 2024-10-24 16:25:20,429 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:95d97106-f45d-48a8-8f04-227b79859a8c 2024-10-24 16:25:38,966 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn-26506286 2024-10-24 16:25:38,976 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-17756595:ctest-vn-26506286:ctest-vn-26506286 2024-10-24 16:25:38,982 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_VrfListReq?name= 2024-10-24 16:25:38,988 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_KRouteReq?x=2 2024-10-24 16:25:38,994 - DEBUG - Unable to fetch route details in vrouter for vrf index 2, Got :[] 2024-10-24 16:25:39,406 - DEBUG - Requesting: http://10.0.0.27:8085/Snh_AgentXmppConnectionStatusReq? 2024-10-24 16:25:48,177 - DEBUG - [10.0.0.27]: Running cmd : hostname 2024-10-24 16:25:48,369 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1 2024-10-24 16:25:48,369 - DEBUG - [10.0.0.27]: Running cmd : hostname -f 2024-10-24 16:25:48,431 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1. 2024-10-24 16:25:48,431 - DEBUG - [10.0.0.27]: Running cmd : docker ps -a 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2024-10-24 16:25:48,571 - DEBUG - Output : NAMES contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_kafka_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 webui_web_1 webui_job_1 config_devicemgr_1 config_schema_1 config_nodemgr_1 config_dnsmasq_1 config_api_1 config_svcmonitor_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:25:48,571 - DEBUG - [10.0.0.27]: Running cmd : getent hosts 10.0.0.27 | head -n 1 | awk '{print $2}' 2024-10-24 16:25:48,638 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1. 2024-10-24 16:25:48,638 - DEBUG - [10.0.0.27]: Running cmd : ip addr show | grep 'inet .*/.* brd ' | awk '{print $2}' 2024-10-24 16:25:48,695 - DEBUG - Output : 10.0.0.27/24 172.17.0.1/16 10.20.0.10/24 2024-10-24 16:25:48,696 - DEBUG - [10.0.0.27]: Running cmd : ip addr show dev vhost0 | grep 'inet .*/.* brd ' | awk '{print $2}' 2024-10-24 16:25:48,760 - DEBUG - Output : 10.20.0.10/24 2024-10-24 16:25:48,760 - DEBUG - [10.0.0.27]: Running cmd : getent hosts 10.20.0.10 | head -n 1 | awk '{print $2}' 2024-10-24 16:25:48,835 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-2143-1-test 2024-10-24 16:25:49,737 - DEBUG - Not creating keypair since it exists 2024-10-24 16:25:50,141 - INFO - Using existing project ['default-domain', 'admin'](a6890942-6a5a-4756-95e0-8a3f393aa814) 2024-10-24 16:25:50,362 - INFO - ================================================================================ 2024-10-24 16:25:50,362 - INFO - STARTING TEST : test_contrail_status 2024-10-24 16:25:50,363 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2024-10-24 16:25:50,488 - DEBUG - Skipping xmpp flap check 2024-10-24 16:25:50,489 - INFO - Initial checks done. Running the testcase now 2024-10-24 16:25:50,489 - INFO - 2024-10-24 16:25:50,489 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:25:50,627 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:25:50,627 - INFO - 10.0.0.27 2024-10-24 16:25:50,627 - INFO - config 2024-10-24 16:25:50,636 - INFO - config-nodemgr:active 2024-10-24 16:25:50,642 - INFO - api-server:active 2024-10-24 16:25:50,648 - INFO - schema:active 2024-10-24 16:25:50,649 - INFO - svc-monitor:inactive 2024-10-24 16:25:50,649 - INFO - device-manager:inactive 2024-10-24 16:25:50,649 - INFO - config-database 2024-10-24 16:25:50,649 - INFO - config-cassandra:active 2024-10-24 16:25:50,649 - INFO - config-zookeeper:active 2024-10-24 16:25:50,649 - INFO - config-rabbitmq:active 2024-10-24 16:25:50,649 - INFO - vrouter 2024-10-24 16:25:50,658 - INFO - vrouter-nodemgr:active 2024-10-24 16:25:50,665 - INFO - agent:active 2024-10-24 16:25:50,665 - INFO - control 2024-10-24 16:25:50,673 - INFO - control-nodemgr:active 2024-10-24 16:25:50,681 - INFO - control:active 2024-10-24 16:25:50,681 - INFO - named:active 2024-10-24 16:25:50,692 - INFO - dns:active 2024-10-24 16:25:50,692 - INFO - analytics 2024-10-24 16:25:50,700 - INFO - analytics-nodemgr:active 2024-10-24 16:25:50,709 - INFO - analytics-api:active 2024-10-24 16:25:50,716 - INFO - collector:active 2024-10-24 16:25:50,716 - INFO - analytics-database 2024-10-24 16:25:50,716 - INFO - analytics-cassandra:active 2024-10-24 16:25:50,724 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:25:50,733 - INFO - query-engine:active 2024-10-24 16:25:50,733 - INFO - webui 2024-10-24 16:25:50,733 - INFO - webui:active 2024-10-24 16:25:50,733 - INFO - webui-middleware:active 2024-10-24 16:25:50,733 - INFO - redis:active 2024-10-24 16:25:50,733 - INFO - analytics_snmp 2024-10-24 16:25:50,739 - INFO - snmp-collector:active 2024-10-24 16:25:50,739 - INFO - snmp-topology:active 2024-10-24 16:25:50,740 - INFO - analytics_alarm 2024-10-24 16:25:50,740 - INFO - alarmgen:active 2024-10-24 16:25:50,740 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:25:50,740 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0 2024-10-24 16:26:00,741 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:26:00,831 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:26:00,832 - INFO - 10.0.0.27 2024-10-24 16:26:00,832 - INFO - config 2024-10-24 16:26:00,839 - INFO - config-nodemgr:active 2024-10-24 16:26:00,845 - INFO - api-server:active 2024-10-24 16:26:00,851 - INFO - schema:active 2024-10-24 16:26:00,851 - INFO - svc-monitor:inactive 2024-10-24 16:26:00,851 - INFO - device-manager:inactive 2024-10-24 16:26:00,851 - INFO - config-database 2024-10-24 16:26:00,851 - INFO - config-cassandra:active 2024-10-24 16:26:00,851 - INFO - config-zookeeper:active 2024-10-24 16:26:00,851 - INFO - config-rabbitmq:active 2024-10-24 16:26:00,851 - INFO - vrouter 2024-10-24 16:26:00,857 - INFO - vrouter-nodemgr:active 2024-10-24 16:26:00,864 - INFO - agent:active 2024-10-24 16:26:00,864 - INFO - control 2024-10-24 16:26:00,870 - INFO - control-nodemgr:active 2024-10-24 16:26:00,876 - INFO - control:active 2024-10-24 16:26:00,877 - INFO - named:active 2024-10-24 16:26:00,886 - INFO - dns:active 2024-10-24 16:26:00,886 - INFO - analytics 2024-10-24 16:26:00,893 - INFO - analytics-nodemgr:active 2024-10-24 16:26:00,899 - INFO - analytics-api:active 2024-10-24 16:26:00,906 - INFO - collector:active 2024-10-24 16:26:00,906 - INFO - analytics-database 2024-10-24 16:26:00,906 - INFO - analytics-cassandra:active 2024-10-24 16:26:00,912 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:26:00,922 - INFO - query-engine:active 2024-10-24 16:26:00,922 - INFO - webui 2024-10-24 16:26:00,922 - INFO - webui:active 2024-10-24 16:26:00,923 - INFO - webui-middleware:active 2024-10-24 16:26:00,923 - INFO - redis:active 2024-10-24 16:26:00,923 - INFO - analytics_snmp 2024-10-24 16:26:00,930 - INFO - snmp-collector:active 2024-10-24 16:26:00,930 - INFO - snmp-topology:active 2024-10-24 16:26:00,930 - INFO - analytics_alarm 2024-10-24 16:26:00,930 - INFO - alarmgen:active 2024-10-24 16:26:00,930 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:26:00,930 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 1 2024-10-24 16:26:10,931 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:26:11,033 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:26:11,033 - INFO - 10.0.0.27 2024-10-24 16:26:11,033 - INFO - config 2024-10-24 16:26:11,043 - INFO - config-nodemgr:active 2024-10-24 16:26:11,050 - INFO - api-server:active 2024-10-24 16:26:11,057 - INFO - schema:active 2024-10-24 16:26:11,058 - INFO - svc-monitor:inactive 2024-10-24 16:26:11,058 - INFO - device-manager:inactive 2024-10-24 16:26:11,058 - INFO - config-database 2024-10-24 16:26:11,058 - INFO - config-cassandra:active 2024-10-24 16:26:11,058 - INFO - config-zookeeper:active 2024-10-24 16:26:11,059 - INFO - config-rabbitmq:active 2024-10-24 16:26:11,059 - INFO - vrouter 2024-10-24 16:26:11,065 - INFO - vrouter-nodemgr:active 2024-10-24 16:26:11,074 - INFO - agent:active 2024-10-24 16:26:11,074 - INFO - control 2024-10-24 16:26:11,081 - INFO - control-nodemgr:active 2024-10-24 16:26:11,091 - INFO - control:active 2024-10-24 16:26:11,091 - INFO - named:active 2024-10-24 16:26:11,102 - INFO - dns:active 2024-10-24 16:26:11,103 - INFO - analytics 2024-10-24 16:26:11,110 - INFO - analytics-nodemgr:active 2024-10-24 16:26:11,117 - INFO - analytics-api:active 2024-10-24 16:26:11,126 - INFO - collector:active 2024-10-24 16:26:11,126 - INFO - analytics-database 2024-10-24 16:26:11,127 - INFO - analytics-cassandra:active 2024-10-24 16:26:11,137 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:26:11,146 - INFO - query-engine:active 2024-10-24 16:26:11,146 - INFO - webui 2024-10-24 16:26:11,146 - INFO - webui:active 2024-10-24 16:26:11,147 - INFO - webui-middleware:active 2024-10-24 16:26:11,147 - INFO - redis:active 2024-10-24 16:26:11,147 - INFO - analytics_snmp 2024-10-24 16:26:11,154 - INFO - snmp-collector:active 2024-10-24 16:26:11,154 - INFO - snmp-topology:active 2024-10-24 16:26:11,154 - INFO - analytics_alarm 2024-10-24 16:26:11,155 - INFO - alarmgen:active 2024-10-24 16:26:11,155 - DEBUG - defaultdict(, {'10.0.0.27': {'svc-monitor': {'status': 'inactive', 'description': None}, 'device-manager': {'status': 'inactive', 'description': None}}}) 2024-10-24 16:26:11,155 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 2 2024-10-24 16:26:21,156 - DEBUG - [10.0.0.27]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2024-10-24 16:26:21,245 - DEBUG - Output : contrail_test_d1ikOrUYc vrouter_vrouter-agent_1 vrouter_nodemgr_1 vrouter_provisioner_1 rsyslogd_rsyslogd_1 analytics_snmp_nodemgr_1 analytics_snmp_topology_1 analytics_snmp_provisioner_1 analytics_snmp_snmp-collector_1 analytics_alarm_alarm-gen_1 analytics_alarm_nodemgr_1 analytics_alarm_provisioner_1 analytics_alarm_kafka_1 analytics_provisioner_1 analytics_collector_1 analytics_nodemgr_1 analytics_api_1 analytics_database_provisioner_1 analytics_database_query-engine_1 analytics_database_nodemgr_1 analytics_database_cassandra_1 control_named_1 control_dns_1 control_control_1 control_nodemgr_1 control_provisioner_1 webui_web_1 webui_job_1 config_devicemgr_1 config_schema_1 config_nodemgr_1 config_provisioner_1 config_dnsmasq_1 config_api_1 config_svcmonitor_1 config_database_rabbitmq_1 config_database_nodemgr_1 config_database_provisioner_1 config_database_cassandra_1 config_database_zookeeper_1 redis_redis_1 barbican_worker barbican_keystone_listener barbican_api horizon heat_engine heat_api_cfn heat_api neutron_server nova_compute nova_libvirt nova_ssh nova_novncproxy nova_conductor nova_api nova_scheduler placement_api glance_api keystone keystone_fernet keystone_ssh rabbitmq memcached mariadb cron kolla_toolbox fluentd 2024-10-24 16:26:21,246 - INFO - 10.0.0.27 2024-10-24 16:26:21,246 - INFO - config 2024-10-24 16:26:21,254 - INFO - config-nodemgr:active 2024-10-24 16:26:21,259 - INFO - api-server:active 2024-10-24 16:26:21,265 - INFO - schema:active 2024-10-24 16:26:21,270 - INFO - svc-monitor:backup 2024-10-24 16:26:21,273 - INFO - device-manager:backup 2024-10-24 16:26:21,273 - INFO - config-database 2024-10-24 16:26:21,273 - INFO - config-cassandra:active 2024-10-24 16:26:21,274 - INFO - config-zookeeper:active 2024-10-24 16:26:21,274 - INFO - config-rabbitmq:active 2024-10-24 16:26:21,274 - INFO - vrouter 2024-10-24 16:26:21,280 - INFO - vrouter-nodemgr:active 2024-10-24 16:26:21,286 - INFO - agent:active 2024-10-24 16:26:21,286 - INFO - control 2024-10-24 16:26:21,292 - INFO - control-nodemgr:active 2024-10-24 16:26:21,301 - INFO - control:active 2024-10-24 16:26:21,301 - INFO - named:active 2024-10-24 16:26:21,307 - INFO - dns:active 2024-10-24 16:26:21,307 - INFO - analytics 2024-10-24 16:26:21,313 - INFO - analytics-nodemgr:active 2024-10-24 16:26:21,319 - INFO - analytics-api:active 2024-10-24 16:26:21,327 - INFO - collector:active 2024-10-24 16:26:21,328 - INFO - analytics-database 2024-10-24 16:26:21,328 - INFO - analytics-cassandra:active 2024-10-24 16:26:21,336 - INFO - analyticsdb-nodemgr:active 2024-10-24 16:26:21,345 - INFO - query-engine:active 2024-10-24 16:26:21,345 - INFO - webui 2024-10-24 16:26:21,345 - INFO - webui:active 2024-10-24 16:26:21,345 - INFO - webui-middleware:active 2024-10-24 16:26:21,345 - INFO - redis:active 2024-10-24 16:26:21,345 - INFO - analytics_snmp 2024-10-24 16:26:21,353 - INFO - snmp-collector:active 2024-10-24 16:26:21,353 - INFO - snmp-topology:active 2024-10-24 16:26:21,353 - INFO - analytics_alarm 2024-10-24 16:26:21,353 - INFO - alarmgen:active 2024-10-24 16:26:21,353 - INFO - All the contrail services are up on all nodes 2024-10-24 16:26:21,353 - INFO - contrail-status passed 2024-10-24 16:26:21,492 - DEBUG - Skipping xmpp flap check 2024-10-24 16:26:21,492 - INFO - END TEST : test_contrail_status : PASSED[0:00:31] 2024-10-24 16:26:21,492 - INFO - --------------------------------------------------------------------------------