2025-10-20 16:33:00,618 - DEBUG - [10.0.0.22]: Running cmd : hostname 2025-10-20 16:33:00,856 - DEBUG - Output : cn-jenkins-deploy-platform-juju-os-600-1 2025-10-20 16:33:00,857 - DEBUG - [10.0.0.22]: Running cmd : hostname -f 2025-10-20 16:33:00,948 - DEBUG - Output : cn-jenkins-deploy-platform-juju-os-600-1 2025-10-20 16:33:00,949 - DEBUG - [10.0.0.22]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2025-10-20 16:33:01,067 - DEBUG - Output : NAMES contrail_test_bZHMkZZoh vrouter-vrouter-agent-1 vrouter-nodemgr-1 webui-job-1 webui-web-1 control-named-1 control-control-1 control-nodemgr-1 control-dns-1 config_database-rabbitmq-1 config_database-cassandra-1 config_database-zookeeper-1 config_database-nodemgr-1 config_api-devicemgr-1 config_api-api-1 config_api-nodemgr-1 config_api-dnsmasq-1 config_api-schema-1 config_api-svcmonitor-1 redis-stunnel-1 redis-redis-1 analytics_snmp-nodemgr-1 analytics_snmp-snmp-collector-1 analytics_snmp-topology-1 analytics_alarm-nodemgr-1 analytics_alarm-alarm-gen-1 analytics_alarm-kafka-1 analytics-nodemgr-1 analytics-collector-1 analytics-api-1 analytics_database-cassandra-1 analytics_database-query-engine-1 analytics_database-nodemgr-1 2025-10-20 16:33:01,067 - DEBUG - [10.0.0.22]: Running cmd : getent hosts 10.0.0.22 | head -n 1 | awk '{print $2}' 2025-10-20 16:33:01,164 - DEBUG - Output : cn-jenkins-deploy-platform-juju-os-600-1 2025-10-20 16:33:01,165 - DEBUG - [10.0.0.22]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2025-10-20 16:33:01,267 - DEBUG - Output : 127.0.0.1/8 252.22.0.1/8 10.226.80.1/24 172.17.0.1/16 10.0.0.22/24 2025-10-20 16:33:01,267 - DEBUG - [10.0.0.22]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2025-10-20 16:33:01,363 - DEBUG - Output : 10.0.0.22/24 2025-10-20 16:33:12,610 - INFO - Using existing project ['admin_domain', 'admin'](eeb09070-0289-47ea-8478-90b170db953d) 2025-10-20 16:33:12,774 - INFO - ================================================================================ 2025-10-20 16:33:12,774 - INFO - STARTING TEST : test_contrail_status 2025-10-20 16:33:12,775 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2025-10-20 16:33:12,978 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:13,032 - DEBUG - Nothing to compare xmpp stats {'10.0.0.22': {'10.0.0.22': '0'}} with 2025-10-20 16:33:13,032 - INFO - Initial checks done. Running the testcase now 2025-10-20 16:33:13,032 - INFO - 2025-10-20 16:33:13,032 - DEBUG - [10.0.0.22]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2025-10-20 16:33:13,107 - DEBUG - Output : contrail_test_bZHMkZZoh control-provisioner-1 config_api-provisioner-1 vrouter-vrouter-agent-1 vrouter-nodemgr-1 vrouter-provisioner-1 webui-job-1 webui-web-1 control-named-1 control-control-1 control-nodemgr-1 control-dns-1 config_database-rabbitmq-1 config_database-cassandra-1 config_database-zookeeper-1 config_database-nodemgr-1 config_database-provisioner-1 config_api-devicemgr-1 config_api-api-1 config_api-nodemgr-1 config_api-dnsmasq-1 config_api-schema-1 config_api-svcmonitor-1 redis-stunnel-1 redis-redis-1 analytics_snmp-nodemgr-1 analytics_snmp-provisioner-1 analytics_snmp-snmp-collector-1 analytics_snmp-topology-1 analytics_alarm-provisioner-1 analytics_alarm-nodemgr-1 analytics_alarm-alarm-gen-1 analytics_alarm-kafka-1 analytics-nodemgr-1 analytics-collector-1 analytics-api-1 analytics-provisioner-1 analytics_database-cassandra-1 analytics_database-provisioner-1 analytics_database-query-engine-1 analytics_database-nodemgr-1 2025-10-20 16:33:13,107 - INFO - 10.0.0.22 2025-10-20 16:33:13,107 - INFO - config 2025-10-20 16:33:13,159 - INFO - config-nodemgr:active 2025-10-20 16:33:13,215 - INFO - api-server:active 2025-10-20 16:33:13,267 - INFO - schema:active 2025-10-20 16:33:13,323 - INFO - svc-monitor:active 2025-10-20 16:33:13,379 - INFO - device-manager:active 2025-10-20 16:33:13,379 - INFO - config-database 2025-10-20 16:33:13,379 - INFO - config-cassandra:active 2025-10-20 16:33:13,379 - INFO - config-zookeeper:active 2025-10-20 16:33:13,379 - INFO - config-rabbitmq:active 2025-10-20 16:33:13,380 - INFO - vrouter 2025-10-20 16:33:13,439 - INFO - vrouter-nodemgr:active 2025-10-20 16:33:13,495 - INFO - agent:active 2025-10-20 16:33:13,495 - INFO - control 2025-10-20 16:33:13,551 - INFO - control-nodemgr:active 2025-10-20 16:33:13,603 - INFO - control:active 2025-10-20 16:33:13,603 - INFO - named:active 2025-10-20 16:33:13,659 - INFO - dns:active 2025-10-20 16:33:13,659 - INFO - analytics 2025-10-20 16:33:13,715 - INFO - analytics-nodemgr:active 2025-10-20 16:33:13,771 - INFO - analytics-api:active 2025-10-20 16:33:13,823 - INFO - collector:active 2025-10-20 16:33:13,823 - INFO - analytics-database 2025-10-20 16:33:13,824 - INFO - analytics-cassandra:active 2025-10-20 16:33:13,879 - INFO - analyticsdb-nodemgr:active 2025-10-20 16:33:13,935 - INFO - query-engine:active 2025-10-20 16:33:13,935 - INFO - webui 2025-10-20 16:33:13,935 - INFO - webui:active 2025-10-20 16:33:13,935 - INFO - webui-middleware:active 2025-10-20 16:33:13,935 - INFO - redis:active 2025-10-20 16:33:13,935 - INFO - analytics_snmp 2025-10-20 16:33:13,987 - INFO - snmp-collector:active 2025-10-20 16:33:13,987 - INFO - snmp-topology:active 2025-10-20 16:33:13,987 - INFO - analytics_alarm 2025-10-20 16:33:13,987 - INFO - alarmgen:active 2025-10-20 16:33:13,987 - INFO - All the contrail services are up on all nodes 2025-10-20 16:33:13,987 - INFO - contrail-status passed 2025-10-20 16:33:14,215 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:14,264 - DEBUG - No XMPP flaps were noticed during the test
2025-10-20 16:33:14,264 - INFO - END TEST : test_contrail_status : PASSED[0:00:02]
2025-10-20 16:33:14,264 - INFO - -------------------------------------------------------------------------------- 2025-10-20 16:33:23,854 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:27,807 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-36317546:ctest-vn22-43346327:ctest-vn22-43346327 2025-10-20 16:33:27,859 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-36317546:ctest-vn22-43346327:ctest-vn22-43346327 2025-10-20 16:33:27,907 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-36317546:ctest-vn22-43346327:ctest-vn22-43346327 2025-10-20 16:33:50,063 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-36317546:ctest-vn22-43346327 2025-10-20 16:33:50,115 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-36317546:ctest-vn22-43346327:ctest-vn22-43346327 2025-10-20 16:33:50,163 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-36317546:ctest-vn22-43346327:ctest-vn22-43346327 2025-10-20 16:33:50,527 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:51,826 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:51,876 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it config_api-api-1 /bin/bash -c 'db-manage check' 2025-10-20 16:33:54,853 - DEBUG - Output : 2025-10-20 16:33:54,658 INFO: (v1.41) Checker check_zk_mode_and_node_count: Success 2025-10-20 16:33:54,710 ERROR: (v1.41) Checker check_cassandra_keyspace_replication: Exception, NoHostAvailable Python 3.9.21: /usr/bin/python3 Mon Oct 20 16:33:54 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={}) 1535 self = args[0] 1536 try: 1537 errors = func(*args, **kwargs) 1538 if not errors: 1539 self._logger.info('(v%s) Checker %s: Success', __version__, errors undefined func = args = (,) kwargs = {} /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_cassandra_keyspace_replication(self=) 1635 raise CassandraConnectionError("error, {}: {}".format( 1636 error, utils.detailed_traceback())) 1637 temp_cluster.connect() 1638 for ks_name, _ in self._db_info: 1639 if self._api_args.cluster_id: temp_cluster = temp_cluster.connect = > /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection._reconnect_internal() NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {'errors': {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}} __dir__ = __doc__ = '\n Raised when an operation is attempted but a...ct, closed, or resulted in errors when used.\n ' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __module__ = 'cassandra.cluster' __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = __weakref__ = None args = ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) errors = {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')} with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication temp_cluster.connect() File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication temp_cluster.connect() File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) NoHostAvailable Python 3.9.21: /usr/bin/python3 Mon Oct 20 16:33:54 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/local/bin/db-manage in 3 from vnc_cfg_api_server.db_manage import main 4 if __name__ == '__main__': 5 if sys.argv[0].endswith('.exe'): 6 sys.argv[0] = sys.argv[0][:-4] 7 sys.exit(main()) sys = sys.exit = main = /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in main() 3451 verb = args.operation 3452 if 'db_%s' % (verb) in globals(): 3453 return globals()['db_%s' % (verb)](args, api_args) 3454 3455 if getattr(DatabaseChecker, verb, None): builtinglobals = verb = 'check' args = Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log') api_args = Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7f1982801fd0>) /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in db_check(args=Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log'), api_args=Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7f1982801fd0>)) 3358 # Mode and node count check across all nodes 3359 db_checker.check_zk_mode_and_node_count() 3360 db_checker.check_cassandra_keyspace_replication() 3361 # Obj UUID cassandra inconsistencies 3362 db_checker.check_obj_mandatory_fields() db_checker = db_checker.check_cassandra_keyspace_replication = > /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={}) 1535 self = args[0] 1536 try: 1537 errors = func(*args, **kwargs) 1538 if not errors: 1539 self._logger.info('(v%s) Checker %s: Success', __version__, errors undefined func = args = (,) kwargs = {} /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_cassandra_keyspace_replication(self=) 1635 raise CassandraConnectionError("error, {}: {}".format( 1636 error, utils.detailed_traceback())) 1637 temp_cluster.connect() 1638 for ks_name, _ in self._db_info: 1639 if self._api_args.cluster_id: temp_cluster = temp_cluster.connect = > /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection._reconnect_internal() NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {'errors': {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}} __dir__ = __doc__ = '\n Raised when an operation is attempted but a...ct, closed, or resulted in errors when used.\n ' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __module__ = 'cassandra.cluster' __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = __weakref__ = None args = ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) errors = {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')} with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/local/bin/db-manage", line 7, in sys.exit(main()) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3453, in main return globals()['db_%s' % (verb)](args, api_args) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3360, in db_check db_checker.check_cassandra_keyspace_replication() File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication temp_cluster.connect() File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.22:9041': ConnectionResetError(104, 'Connection reset by peer')}) 2025-10-20 16:33:55,253 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:55,597 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:55,871 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:56,146 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:56,411 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:33:59,637 - INFO - admin user already present 2025-10-20 16:34:00,694 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:34:04,343 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:04,395 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:04,447 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:06,647 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846:ctest-vn1-76439846 2025-10-20 16:34:06,699 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846:ctest-vn1-76439846 2025-10-20 16:34:06,747 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846:ctest-vn1-76439846 2025-10-20 16:34:21,410 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a27c51f5-73ba-457b-aa6c-365b914cd96d 2025-10-20 16:34:21,465 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846:ctest-vn1-76439846 2025-10-20 16:34:21,525 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a27c51f5-73ba-457b-aa6c-365b914cd96d 2025-10-20 16:34:37,628 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:8780bd55-3b38-42fc-a4c2-f246be8010ec 2025-10-20 16:34:37,683 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:37,731 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:8780bd55-3b38-42fc-a4c2-f246be8010ec 2025-10-20 16:34:39,520 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a27c51f5-73ba-457b-aa6c-365b914cd96d 2025-10-20 16:34:39,576 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483 2025-10-20 16:34:39,635 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:39,687 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=3 2025-10-20 16:34:46,207 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846 2025-10-20 16:34:46,255 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846:ctest-vn1-76439846 2025-10-20 16:34:46,303 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-vn1-76439846:ctest-vn1-76439846 2025-10-20 16:34:46,887 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483 2025-10-20 16:34:46,939 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:46,991 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-64767379:ctest-fvn-45350483:ctest-fvn-45350483 2025-10-20 16:34:47,473 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:34:51,197 - INFO - ctest-TestHeat-59049792 user already added to project 2025-10-20 16:34:51,541 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:34:52,465 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:00,552 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:08,988 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:12,974 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:15,888 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:16,292 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:25,409 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ffbf712a-77ff-4489-8e40-43e78a381f8c 2025-10-20 16:35:25,459 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestPorts-23468022:ctest-vn1-81036112:ctest-vn1-81036112 2025-10-20 16:35:25,511 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ffbf712a-77ff-4489-8e40-43e78a381f8c 2025-10-20 16:35:39,092 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:8bfa0707-6a7b-45a2-b044-80ebcdfa8cc3 2025-10-20 16:35:39,143 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestPorts-23468022:ctest-vn1-81036112:ctest-vn1-81036112 2025-10-20 16:35:39,191 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:8bfa0707-6a7b-45a2-b044-80ebcdfa8cc3 2025-10-20 16:35:46,492 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:35:50,573 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:36:01,936 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:29c3f558-91ac-49d6-af22-44a940c0bc0a 2025-10-20 16:36:01,987 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRouters-47569119:ctest-vn1-30068744:ctest-vn1-30068744 2025-10-20 16:36:02,035 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:29c3f558-91ac-49d6-af22-44a940c0bc0a 2025-10-20 16:36:08,552 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:bb46bf21-2025-483b-8cf2-53a4391ac119 2025-10-20 16:36:08,603 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRouters-47569119:ctest-vn2-36888203:ctest-vn2-36888203 2025-10-20 16:36:08,651 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:bb46bf21-2025-483b-8cf2-53a4391ac119 2025-10-20 16:36:31,585 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:36:36,077 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:36:49,628 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4005abf9-0420-4f40-b6d0-77d70799ae0e 2025-10-20 16:36:49,679 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRoutersBasic-66312010:ctest-vn-08007384:ctest-vn-08007384 2025-10-20 16:36:49,731 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4005abf9-0420-4f40-b6d0-77d70799ae0e 2025-10-20 16:36:56,116 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:3e2c600f-3f31-435a-b5d2-db6e2a6d19c1 2025-10-20 16:36:56,167 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRoutersBasic-66312010:ctest-ext_vn-77794599:ctest-ext_vn-77794599 2025-10-20 16:36:56,219 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:3e2c600f-3f31-435a-b5d2-db6e2a6d19c1 2025-10-20 16:37:05,597 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:09,540 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:15,575 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-04906287:ctest-vn-04906287 2025-10-20 16:37:15,623 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-04906287:ctest-vn-04906287 2025-10-20 16:37:15,672 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-04906287:ctest-vn-04906287 2025-10-20 16:37:16,959 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-54481223:ctest-vn-54481223 2025-10-20 16:37:17,011 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-54481223:ctest-vn-54481223 2025-10-20 16:37:17,059 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-54481223:ctest-vn-54481223 2025-10-20 16:37:18,139 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-54481223 2025-10-20 16:37:18,191 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-54481223:ctest-vn-54481223 2025-10-20 16:37:18,239 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-54481223:ctest-vn-54481223 2025-10-20 16:37:18,787 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-04906287 2025-10-20 16:37:18,835 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-04906287:ctest-vn-04906287 2025-10-20 16:37:18,883 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-53021743:ctest-vn-04906287:ctest-vn-04906287 2025-10-20 16:37:19,380 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:23,157 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:23,545 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:27,911 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-45060209:ctest-vn4-08631101:ctest-vn4-08631101 2025-10-20 16:37:27,959 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-45060209:ctest-vn4-08631101:ctest-vn4-08631101 2025-10-20 16:37:28,007 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-45060209:ctest-vn4-08631101:ctest-vn4-08631101 2025-10-20 16:37:28,756 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-45060209:ctest-vn4-08631101 2025-10-20 16:37:28,807 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-45060209:ctest-vn4-08631101:ctest-vn4-08631101 2025-10-20 16:37:28,855 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-45060209:ctest-vn4-08631101:ctest-vn4-08631101 2025-10-20 16:37:29,541 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:29,947 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:33,765 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:37:38,843 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194:ctest-vn-86027194 2025-10-20 16:37:38,891 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194:ctest-vn-86027194 2025-10-20 16:37:38,939 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194:ctest-vn-86027194 2025-10-20 16:37:40,223 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102:ctest-vn-54047102 2025-10-20 16:37:40,275 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102:ctest-vn-54047102 2025-10-20 16:37:40,323 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102:ctest-vn-54047102 2025-10-20 16:37:48,637 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:040e289f-8422-4850-8c6b-2c97e32f0568 2025-10-20 16:37:48,691 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194:ctest-vn-86027194 2025-10-20 16:37:48,747 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:040e289f-8422-4850-8c6b-2c97e32f0568 2025-10-20 16:37:55,092 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:36f189ce-1b5d-43e2-b8b9-452159397cd1 2025-10-20 16:37:55,140 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102:ctest-vn-54047102 2025-10-20 16:37:55,195 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:36f189ce-1b5d-43e2-b8b9-452159397cd1 2025-10-20 16:38:07,412 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102 2025-10-20 16:38:07,463 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102:ctest-vn-54047102 2025-10-20 16:38:07,511 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-54047102:ctest-vn-54047102 2025-10-20 16:38:08,064 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194 2025-10-20 16:38:08,115 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194:ctest-vn-86027194 2025-10-20 16:38:08,163 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-86523354:ctest-vn-86027194:ctest-vn-86027194 2025-10-20 16:38:08,644 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:38:12,484 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:38:28,557 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:bddd98f0-e840-4bc5-b391-f6ad86ccb141 2025-10-20 16:38:28,614 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRP-25495611:ctest-bgpaas_vn-10538500:ctest-bgpaas_vn-10538500 2025-10-20 16:38:28,667 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:bddd98f0-e840-4bc5-b391-f6ad86ccb141 2025-10-20 16:38:39,280 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:90b82769-9a18-4da1-a575-80fd82c942ab 2025-10-20 16:38:39,338 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRP-25495611:ctest-bgpaas_vn-86735735:ctest-bgpaas_vn-86735735 2025-10-20 16:38:39,391 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:90b82769-9a18-4da1-a575-80fd82c942ab 2025-10-20 16:38:41,238 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:38:48,309 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:38:52,261 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:38:55,643 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:38:55,695 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:38:55,747 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:38:57,847 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:38:57,903 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:38:57,951 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:39:00,091 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:39:00,139 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:39:00,187 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:39:17,532 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:e3279ca0-b3fc-47c9-9380-1e4870b9a951 2025-10-20 16:39:17,584 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:39:17,631 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dd01db23-5cad-4a32-837a-f92cf2f8d1a0 2025-10-20 16:39:17,683 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:39:17,731 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:037d18ca-b785-45e3-a4ba-40d2283dfe00 2025-10-20 16:39:17,779 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:39:17,831 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:037d18ca-b785-45e3-a4ba-40d2283dfe00 2025-10-20 16:39:17,883 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dd01db23-5cad-4a32-837a-f92cf2f8d1a0 2025-10-20 16:39:17,931 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:e3279ca0-b3fc-47c9-9380-1e4870b9a951 2025-10-20 16:39:54,996 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ce27183f-108a-4194-ba37-e8ae87c7e878 2025-10-20 16:39:55,047 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:39:55,099 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ce27183f-108a-4194-ba37-e8ae87c7e878 2025-10-20 16:39:57,284 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dbbb54e5-9376-4ac7-8908-72c4bf2cf8cc 2025-10-20 16:39:57,339 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:39:57,391 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dbbb54e5-9376-4ac7-8908-72c4bf2cf8cc 2025-10-20 16:40:04,177 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:037d18ca-b785-45e3-a4ba-40d2283dfe00 2025-10-20 16:40:04,231 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dd01db23-5cad-4a32-837a-f92cf2f8d1a0 2025-10-20 16:40:04,283 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:e3279ca0-b3fc-47c9-9380-1e4870b9a951 2025-10-20 16:40:05,619 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446 2025-10-20 16:40:05,671 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dd01db23-5cad-4a32-837a-f92cf2f8d1a0 2025-10-20 16:40:05,724 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-20 16:40:05,740 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:40:05,946 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=2 2025-10-20 16:40:06,186 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Layer2RouteReq?x=2 2025-10-20 16:40:06,203 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446 2025-10-20 16:40:06,260 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:40:06,466 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=2 2025-10-20 16:40:06,699 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Layer2RouteReq?x=2 2025-10-20 16:40:06,724 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310 2025-10-20 16:40:06,780 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:e3279ca0-b3fc-47c9-9380-1e4870b9a951 2025-10-20 16:40:06,840 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-20 16:40:06,908 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:40:07,130 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=4 2025-10-20 16:40:07,371 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Layer2RouteReq?x=4 2025-10-20 16:40:07,383 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310 2025-10-20 16:40:07,431 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:40:07,638 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=4 2025-10-20 16:40:07,850 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Layer2RouteReq?x=4 2025-10-20 16:40:07,865 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351 2025-10-20 16:40:07,919 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:037d18ca-b785-45e3-a4ba-40d2283dfe00 2025-10-20 16:40:07,967 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-20 16:40:07,977 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:40:08,178 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=3 2025-10-20 16:40:08,394 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Layer2RouteReq?x=3 2025-10-20 16:40:08,408 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351 2025-10-20 16:40:08,455 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:40:08,654 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=3 2025-10-20 16:40:08,866 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Layer2RouteReq?x=3 2025-10-20 16:40:09,590 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name= 2025-10-20 16:40:09,648 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=5 2025-10-20 16:40:09,701 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:dd01db23-5cad-4a32-837a-f92cf2f8d1a0 2025-10-20 16:40:09,874 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name= 2025-10-20 16:40:09,923 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_Inet4UcRouteReq?x=6 2025-10-20 16:40:09,973 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:037d18ca-b785-45e3-a4ba-40d2283dfe00 2025-10-20 16:40:18,107 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351 2025-10-20 16:40:18,159 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:40:18,207 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-right_vn-29941351:ctest-right_vn-29941351 2025-10-20 16:40:18,743 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446 2025-10-20 16:40:18,795 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:40:18,843 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-left_vn-34594446:ctest-left_vn-34594446 2025-10-20 16:40:19,411 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310 2025-10-20 16:40:19,459 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:40:19,507 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-85882053:ctest-mgmt_vn-37316310:ctest-mgmt_vn-37316310 2025-10-20 16:40:19,988 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:40:23,817 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:40:33,912 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ffd33cfa-3987-47b4-a67d-be43f79c14de 2025-10-20 16:40:33,964 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestvDNSBasic0-17229487:ctest-vn1-vdns-52087209:ctest-vn1-vdns-52087209 2025-10-20 16:40:34,011 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ffd33cfa-3987-47b4-a67d-be43f79c14de 2025-10-20 16:40:49,320 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_DnsInfo? 2025-10-20 16:40:56,444 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:0abfaea8-64c0-41b4-9a24-40eab581a68b 2025-10-20 16:40:56,491 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestvDNSBasic0-17229487:ctest-vn1-vdns-52087209:ctest-vn1-vdns-52087209 2025-10-20 16:40:56,539 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:0abfaea8-64c0-41b4-9a24-40eab581a68b 2025-10-20 16:41:13,787 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_DnsInfo? 2025-10-20 16:41:25,845 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:41:29,629 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:41:39,529 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:c128bf20-ce33-4653-9140-3e0432ef42c6 2025-10-20 16:41:39,580 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVNx-57883831:ctest-vn-05057082:ctest-vn-05057082 2025-10-20 16:41:39,631 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:c128bf20-ce33-4653-9140-3e0432ef42c6 2025-10-20 16:41:46,004 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:746155b3-7ff3-4931-9609-4e6e51a9f053 2025-10-20 16:41:46,055 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVNx-57883831:ctest-vn-05057082:ctest-vn-05057082 2025-10-20 16:41:46,103 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:746155b3-7ff3-4931-9609-4e6e51a9f053 2025-10-20 16:41:55,208 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:41:59,049 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:42:07,006 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it vrouter-vrouter-agent-1 /bin/bash -c 'echo "10.0.0.22 cn-jenkins-deploy-platform-juju-os-600-1-test" >> /etc/hosts' 2025-10-20 16:42:07,155 - DEBUG - Output : 2025-10-20 16:42:07,896 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d7038747-5138-4e99-b846-3292f9b12422 2025-10-20 16:42:07,948 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn2_metadata-38022475:ctest-vn2_metadata-38022475 2025-10-20 16:42:07,999 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d7038747-5138-4e99-b846-3292f9b12422 2025-10-20 16:42:20,649 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it vrouter-vrouter-agent-1 /bin/bash -c 'sed -i '$ d' /etc/hosts' 2025-10-20 16:42:20,829 - DEBUG - Output : sed: -e expression #1, char 1: missing command 2025-10-20 16:42:23,693 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:42:24,081 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:42:31,788 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:1755b363-48ac-4149-8e32-d4f0008ae0c6 2025-10-20 16:42:31,836 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-40443157:ctest-vn-40443157 2025-10-20 16:42:31,883 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:1755b363-48ac-4149-8e32-d4f0008ae0c6 2025-10-20 16:42:46,283 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:42:46,688 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:42:54,244 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-49321902:ctest-vn-49321902 2025-10-20 16:42:54,848 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ee31477c-0657-4461-9570-7b3dbc2e5a07 2025-10-20 16:42:54,899 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-49321902:ctest-vn-49321902 2025-10-20 16:42:54,947 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ee31477c-0657-4461-9570-7b3dbc2e5a07 2025-10-20 16:43:05,472 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9e69ec5d-528a-4cc5-a28d-174c005d9303 2025-10-20 16:43:05,519 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-49321902:ctest-vn-49321902 2025-10-20 16:43:05,567 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9e69ec5d-528a-4cc5-a28d-174c005d9303 2025-10-20 16:43:07,744 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:8629e091-9e58-4f70-888d-da8cad42f120 2025-10-20 16:43:07,799 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-49321902:ctest-vn-49321902 2025-10-20 16:43:07,855 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:8629e091-9e58-4f70-888d-da8cad42f120 2025-10-20 16:43:26,791 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-49321902 2025-10-20 16:43:26,843 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-56316307:ctest-vn-49321902:ctest-vn-49321902 2025-10-20 16:43:26,891 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_VrfListReq?name= 2025-10-20 16:43:26,939 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_KRouteReq?x=2 2025-10-20 16:43:26,991 - DEBUG - Unable to fetch route details in vrouter for vrf index 2, Got :[] 2025-10-20 16:43:27,621 - DEBUG - Requesting: https://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-20 16:45:24,736 - INFO - Using existing project ['admin_domain', 'admin'](eeb09070-0289-47ea-8478-90b170db953d) 2025-10-20 16:45:24,745 - INFO - ================================================================================ 2025-10-20 16:45:24,745 - INFO - STARTING TEST : test_contrail_status 2025-10-20 16:45:24,745 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2025-10-20 16:45:24,958 - DEBUG - Skipping xmpp flap check 2025-10-20 16:45:24,958 - INFO - Initial checks done. Running the testcase now 2025-10-20 16:45:24,958 - INFO - 2025-10-20 16:45:25,080 - INFO - 10.0.0.22 2025-10-20 16:45:25,080 - INFO - config 2025-10-20 16:45:25,135 - INFO - config-nodemgr:active 2025-10-20 16:45:25,215 - INFO - api-server:active 2025-10-20 16:45:25,275 - INFO - schema:active 2025-10-20 16:45:25,327 - INFO - svc-monitor:active 2025-10-20 16:45:25,383 - INFO - device-manager:active 2025-10-20 16:45:25,383 - INFO - config-database 2025-10-20 16:45:25,383 - INFO - config-cassandra:active 2025-10-20 16:45:25,383 - INFO - config-zookeeper:active 2025-10-20 16:45:25,383 - INFO - config-rabbitmq:active 2025-10-20 16:45:25,383 - INFO - vrouter 2025-10-20 16:45:25,435 - INFO - vrouter-nodemgr:active 2025-10-20 16:45:25,487 - INFO - agent:active 2025-10-20 16:45:25,487 - INFO - control 2025-10-20 16:45:25,547 - INFO - control-nodemgr:active 2025-10-20 16:45:25,651 - INFO - control:active 2025-10-20 16:45:25,651 - INFO - named:active 2025-10-20 16:45:25,711 - INFO - dns:active 2025-10-20 16:45:25,711 - INFO - analytics 2025-10-20 16:45:25,767 - INFO - analytics-nodemgr:active 2025-10-20 16:45:25,823 - INFO - analytics-api:active 2025-10-20 16:45:25,887 - INFO - collector:active 2025-10-20 16:45:25,887 - INFO - analytics-database 2025-10-20 16:45:25,887 - INFO - analytics-cassandra:active 2025-10-20 16:45:25,947 - INFO - analyticsdb-nodemgr:active 2025-10-20 16:45:26,051 - INFO - query-engine:active 2025-10-20 16:45:26,051 - INFO - webui 2025-10-20 16:45:26,051 - INFO - webui:active 2025-10-20 16:45:26,051 - INFO - webui-middleware:active 2025-10-20 16:45:26,051 - INFO - redis:active 2025-10-20 16:45:26,051 - INFO - analytics_snmp 2025-10-20 16:45:26,107 - INFO - snmp-collector:active 2025-10-20 16:45:26,107 - INFO - snmp-topology:active 2025-10-20 16:45:26,107 - INFO - analytics_alarm 2025-10-20 16:45:26,107 - INFO - alarmgen:active 2025-10-20 16:45:26,107 - INFO - All the contrail services are up on all nodes 2025-10-20 16:45:26,107 - INFO - contrail-status passed 2025-10-20 16:45:26,318 - DEBUG - Skipping xmpp flap check
2025-10-20 16:45:26,318 - INFO - END TEST : test_contrail_status : PASSED[0:00:02]
2025-10-20 16:45:26,319 - INFO - --------------------------------------------------------------------------------