2025-10-22 14:56:08,839 - DEBUG - [10.0.0.241]: Running cmd : hostname 2025-10-22 14:56:09,612 - DEBUG - Output : cn-jenkins-deploy-platform-juju-os-601-1 2025-10-22 14:56:09,612 - DEBUG - [10.0.0.241]: Running cmd : hostname -f 2025-10-22 14:56:09,704 - DEBUG - Output : cn-jenkins-deploy-platform-juju-os-601-1 2025-10-22 14:56:09,704 - DEBUG - [10.0.0.241]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2025-10-22 14:56:09,827 - DEBUG - Output : NAMES contrail_test_nrzngHwC0 vrouter-vrouter-agent-1 vrouter-nodemgr-1 webui-job-1 webui-web-1 analytics_snmp-snmp-collector-1 analytics_snmp-nodemgr-1 analytics_snmp-topology-1 analytics_alarm-nodemgr-1 analytics_alarm-alarm-gen-1 analytics_alarm-kafka-1 analytics-api-1 analytics-nodemgr-1 analytics-collector-1 analytics_database-query-engine-1 analytics_database-cassandra-1 analytics_database-nodemgr-1 redis-stunnel-1 redis-redis-1 control-named-1 control-nodemgr-1 control-dns-1 control-control-1 config_database-zookeeper-1 config_database-nodemgr-1 config_database-rabbitmq-1 config_database-cassandra-1 config_api-devicemgr-1 config_api-schema-1 config_api-dnsmasq-1 config_api-svcmonitor-1 config_api-nodemgr-1 config_api-api-1 2025-10-22 14:56:09,827 - DEBUG - [10.0.0.241]: Running cmd : getent hosts 10.0.0.241 | head -n 1 | awk '{print $2}' 2025-10-22 14:56:09,920 - DEBUG - Output : cn-jenkins-deploy-platform-juju-os-601-1 2025-10-22 14:56:09,920 - DEBUG - [10.0.0.241]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2025-10-22 14:56:10,016 - DEBUG - Output : 127.0.0.1/8 252.241.0.1/8 10.29.241.1/24 172.17.0.1/16 10.0.0.241/24 2025-10-22 14:56:10,017 - DEBUG - [10.0.0.241]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2025-10-22 14:56:10,113 - DEBUG - Output : 10.0.0.241/24 2025-10-22 14:56:21,005 - INFO - Using existing project ['admin_domain', 'admin'](9de6d401-db5c-425c-b65a-d62f9ed1f01c) 2025-10-22 14:56:21,167 - INFO - ================================================================================ 2025-10-22 14:56:21,167 - INFO - STARTING TEST : test_contrail_status 2025-10-22 14:56:21,167 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2025-10-22 14:56:21,371 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:56:21,428 - DEBUG - Nothing to compare xmpp stats {'10.0.0.241': {'10.0.0.241': '0'}} with 2025-10-22 14:56:21,428 - INFO - Initial checks done. Running the testcase now 2025-10-22 14:56:21,428 - INFO - 2025-10-22 14:56:21,428 - DEBUG - [10.0.0.241]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null 2025-10-22 14:56:21,516 - DEBUG - Output : contrail_test_nrzngHwC0 vrouter-vrouter-agent-1 vrouter-nodemgr-1 vrouter-provisioner-1 webui-job-1 webui-web-1 control-provisioner-1 config_api-provisioner-1 analytics_snmp-provisioner-1 analytics_snmp-snmp-collector-1 analytics_snmp-nodemgr-1 analytics_snmp-topology-1 analytics_alarm-nodemgr-1 analytics_alarm-alarm-gen-1 analytics_alarm-kafka-1 analytics_alarm-provisioner-1 analytics-provisioner-1 analytics-api-1 analytics-nodemgr-1 analytics-collector-1 analytics_database-query-engine-1 analytics_database-cassandra-1 analytics_database-nodemgr-1 analytics_database-provisioner-1 redis-stunnel-1 redis-redis-1 control-named-1 control-nodemgr-1 control-dns-1 control-control-1 config_database-zookeeper-1 config_database-nodemgr-1 config_database-rabbitmq-1 config_database-cassandra-1 config_database-provisioner-1 config_api-devicemgr-1 config_api-schema-1 config_api-dnsmasq-1 config_api-svcmonitor-1 config_api-nodemgr-1 config_api-api-1 2025-10-22 14:56:21,516 - INFO - 10.0.0.241 2025-10-22 14:56:21,516 - INFO - config 2025-10-22 14:56:21,571 - INFO - config-nodemgr:active 2025-10-22 14:56:21,627 - INFO - api-server:active 2025-10-22 14:56:21,683 - INFO - schema:active 2025-10-22 14:56:21,735 - INFO - svc-monitor:active 2025-10-22 14:56:21,791 - INFO - device-manager:active 2025-10-22 14:56:21,791 - INFO - config-database 2025-10-22 14:56:21,791 - INFO - config-cassandra:active 2025-10-22 14:56:21,791 - INFO - config-zookeeper:active 2025-10-22 14:56:21,791 - INFO - config-rabbitmq:active 2025-10-22 14:56:21,791 - INFO - vrouter 2025-10-22 14:56:21,847 - INFO - vrouter-nodemgr:active 2025-10-22 14:56:21,911 - INFO - agent:active 2025-10-22 14:56:21,912 - INFO - control 2025-10-22 14:56:21,967 - INFO - control-nodemgr:active 2025-10-22 14:56:22,023 - INFO - control:active 2025-10-22 14:56:22,023 - INFO - named:active 2025-10-22 14:56:22,079 - INFO - dns:active 2025-10-22 14:56:22,079 - INFO - analytics 2025-10-22 14:56:22,139 - INFO - analytics-nodemgr:active 2025-10-22 14:56:22,191 - INFO - analytics-api:active 2025-10-22 14:56:22,243 - INFO - collector:active 2025-10-22 14:56:22,244 - INFO - analytics-database 2025-10-22 14:56:22,244 - INFO - analytics-cassandra:active 2025-10-22 14:56:22,299 - INFO - analyticsdb-nodemgr:active 2025-10-22 14:56:22,355 - INFO - query-engine:active 2025-10-22 14:56:22,355 - INFO - webui 2025-10-22 14:56:22,355 - INFO - webui:active 2025-10-22 14:56:22,355 - INFO - webui-middleware:active 2025-10-22 14:56:22,355 - INFO - redis:active 2025-10-22 14:56:22,355 - INFO - analytics_snmp 2025-10-22 14:56:22,411 - INFO - snmp-collector:active 2025-10-22 14:56:22,411 - INFO - snmp-topology:active 2025-10-22 14:56:22,411 - INFO - analytics_alarm 2025-10-22 14:56:22,411 - INFO - alarmgen:active 2025-10-22 14:56:22,411 - INFO - All the contrail services are up on all nodes 2025-10-22 14:56:22,411 - INFO - contrail-status passed 2025-10-22 14:56:22,622 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:56:22,671 - DEBUG - No XMPP flaps were noticed during the test
2025-10-22 14:56:22,672 - INFO - END TEST : test_contrail_status : PASSED[0:00:01]
2025-10-22 14:56:22,672 - INFO - -------------------------------------------------------------------------------- 2025-10-22 14:56:32,795 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:56:36,900 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-72577524:ctest-vn22-61007283:ctest-vn22-61007283 2025-10-22 14:56:36,951 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-72577524:ctest-vn22-61007283:ctest-vn22-61007283 2025-10-22 14:56:37,003 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-72577524:ctest-vn22-61007283:ctest-vn22-61007283 2025-10-22 14:57:03,111 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-72577524:ctest-vn22-61007283 2025-10-22 14:57:03,167 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-72577524:ctest-vn22-61007283:ctest-vn22-61007283 2025-10-22 14:57:03,215 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-AnalyticsBasicTestSanity-72577524:ctest-vn22-61007283:ctest-vn22-61007283 2025-10-22 14:57:03,579 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:04,923 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:04,976 - DEBUG - [10.0.0.241]: Running cmd : docker exec --privileged -it config_api-api-1 /bin/bash -c 'db-manage check' 2025-10-22 14:57:07,884 - DEBUG - Output : 2025-10-22 14:57:07,673 INFO: (v1.41) Checker check_zk_mode_and_node_count: Success 2025-10-22 14:57:07,749 ERROR: (v1.41) Checker check_cassandra_keyspace_replication: Exception, NoHostAvailable Python 3.9.21: /usr/bin/python3 Wed Oct 22 14:57:07 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={}) 1535 self = args[0] 1536 try: 1537 errors = func(*args, **kwargs) 1538 if not errors: 1539 self._logger.info('(v%s) Checker %s: Success', __version__, errors undefined func = args = (,) kwargs = {} /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_cassandra_keyspace_replication(self=) 1635 raise CassandraConnectionError("error, {}: {}".format( 1636 error, utils.detailed_traceback())) 1637 temp_cluster.connect() 1638 for ks_name, _ in self._db_info: 1639 if self._api_args.cluster_id: temp_cluster = temp_cluster.connect = > /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection._reconnect_internal() NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {'errors': {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}} __dir__ = __doc__ = '\n Raised when an operation is attempted but a...ct, closed, or resulted in errors when used.\n ' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __module__ = 'cassandra.cluster' __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = __weakref__ = None args = ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) errors = {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')} with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication temp_cluster.connect() File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication temp_cluster.connect() File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) NoHostAvailable Python 3.9.21: /usr/bin/python3 Wed Oct 22 14:57:07 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/local/bin/db-manage in 3 from vnc_cfg_api_server.db_manage import main 4 if __name__ == '__main__': 5 if sys.argv[0].endswith('.exe'): 6 sys.argv[0] = sys.argv[0][:-4] 7 sys.exit(main()) sys = sys.exit = main = /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in main() 3451 verb = args.operation 3452 if 'db_%s' % (verb) in globals(): 3453 return globals()['db_%s' % (verb)](args, api_args) 3454 3455 if getattr(DatabaseChecker, verb, None): builtinglobals = verb = 'check' args = Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log') api_args = Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7f06ac252fd0>) /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in db_check(args=Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log'), api_args=Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7f06ac252fd0>)) 3358 # Mode and node count check across all nodes 3359 db_checker.check_zk_mode_and_node_count() 3360 db_checker.check_cassandra_keyspace_replication() 3361 # Obj UUID cassandra inconsistencies 3362 db_checker.check_obj_mandatory_fields() db_checker = db_checker.check_cassandra_keyspace_replication = > /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={}) 1535 self = args[0] 1536 try: 1537 errors = func(*args, **kwargs) 1538 if not errors: 1539 self._logger.info('(v%s) Checker %s: Success', __version__, errors undefined func = args = (,) kwargs = {} /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_cassandra_keyspace_replication(self=) 1635 raise CassandraConnectionError("error, {}: {}".format( 1636 error, utils.detailed_traceback())) 1637 temp_cluster.connect() 1638 for ks_name, _ in self._db_info: 1639 if self._api_args.cluster_id: temp_cluster = temp_cluster.connect = > /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.Cluster.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection.connect() /cassandra/cluster.py in cassandra.cluster.ControlConnection._reconnect_internal() NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {'errors': {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}} __dir__ = __doc__ = '\n Raised when an operation is attempted but a...ct, closed, or resulted in errors when used.\n ' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __module__ = 'cassandra.cluster' __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = __weakref__ = None args = ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) errors = {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')} with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/local/bin/db-manage", line 7, in sys.exit(main()) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3453, in main return globals()['db_%s' % (verb)](args, api_args) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3360, in db_check db_checker.check_cassandra_keyspace_replication() File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication temp_cluster.connect() File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.241:9041': ConnectionResetError(104, 'Connection reset by peer')}) 2025-10-22 14:57:08,146 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:08,487 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:08,790 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:09,042 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:09,358 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:12,634 - INFO - admin user already present 2025-10-22 14:57:13,634 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:17,747 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:17,795 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:17,847 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:20,051 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536:ctest-vn1-33244536 2025-10-22 14:57:20,099 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536:ctest-vn1-33244536 2025-10-22 14:57:20,147 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536:ctest-vn1-33244536 2025-10-22 14:57:30,708 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:39fe9b24-edad-4fb6-b5ac-2e304e707467 2025-10-22 14:57:30,760 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536:ctest-vn1-33244536 2025-10-22 14:57:30,811 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:39fe9b24-edad-4fb6-b5ac-2e304e707467 2025-10-22 14:57:37,661 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:6f753dd9-0140-4d82-be8d-d4c4e0e53ae3 2025-10-22 14:57:37,711 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:37,763 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:6f753dd9-0140-4d82-be8d-d4c4e0e53ae3 2025-10-22 14:57:39,536 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:39fe9b24-edad-4fb6-b5ac-2e304e707467 2025-10-22 14:57:39,588 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172 2025-10-22 14:57:39,644 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:39,695 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=3 2025-10-22 14:57:46,128 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536 2025-10-22 14:57:46,179 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536:ctest-vn1-33244536 2025-10-22 14:57:46,231 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-vn1-33244536:ctest-vn1-33244536 2025-10-22 14:57:46,808 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172 2025-10-22 14:57:46,855 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:46,907 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-FloatingipBasicTestSanity-66455798:ctest-fvn-73785172:ctest-fvn-73785172 2025-10-22 14:57:47,413 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:51,131 - INFO - ctest-TestHeat-48808912 user already added to project 2025-10-22 14:57:51,469 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:57:52,425 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:01,284 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:09,609 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:13,489 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:16,522 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:16,912 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:27,757 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d4e15be6-0cb9-4f1e-8cce-be98050fb73c 2025-10-22 14:58:27,808 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestPorts-14058334:ctest-vn1-00258412:ctest-vn1-00258412 2025-10-22 14:58:27,859 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d4e15be6-0cb9-4f1e-8cce-be98050fb73c 2025-10-22 14:58:36,220 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ee6d8446-3832-4e8f-9da5-0c312104c7a6 2025-10-22 14:58:36,275 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestPorts-14058334:ctest-vn1-00258412:ctest-vn1-00258412 2025-10-22 14:58:36,323 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ee6d8446-3832-4e8f-9da5-0c312104c7a6 2025-10-22 14:58:42,373 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:46,509 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:58:58,716 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7e5faadc-15d6-45ae-9fcc-aecb37f37ffc 2025-10-22 14:58:58,779 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRouters-84285039:ctest-vn1-12465446:ctest-vn1-12465446 2025-10-22 14:58:58,832 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7e5faadc-15d6-45ae-9fcc-aecb37f37ffc 2025-10-22 14:59:05,374 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:568d6c79-b120-4201-b164-645ff4d79a38 2025-10-22 14:59:05,428 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRouters-84285039:ctest-vn2-13392565:ctest-vn2-13392565 2025-10-22 14:59:05,479 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:568d6c79-b120-4201-b164-645ff4d79a38 2025-10-22 14:59:28,678 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:59:33,847 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 14:59:47,784 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:2abac760-e09d-457c-99e7-d6912a428753 2025-10-22 14:59:47,840 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRoutersBasic-74818992:ctest-vn-02672365:ctest-vn-02672365 2025-10-22 14:59:47,898 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:2abac760-e09d-457c-99e7-d6912a428753 2025-10-22 14:59:54,385 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:277697f7-6214-456f-aab7-50bb4a5332ad 2025-10-22 14:59:54,435 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRoutersBasic-74818992:ctest-ext_vn-95871401:ctest-ext_vn-95871401 2025-10-22 14:59:54,483 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:277697f7-6214-456f-aab7-50bb4a5332ad 2025-10-22 15:00:03,526 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:07,674 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:13,615 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-61299989:ctest-vn-61299989 2025-10-22 15:00:13,663 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-61299989:ctest-vn-61299989 2025-10-22 15:00:13,715 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-61299989:ctest-vn-61299989 2025-10-22 15:00:14,999 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-10092693:ctest-vn-10092693 2025-10-22 15:00:15,047 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-10092693:ctest-vn-10092693 2025-10-22 15:00:15,099 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-10092693:ctest-vn-10092693 2025-10-22 15:00:16,192 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-10092693 2025-10-22 15:00:16,243 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-10092693:ctest-vn-10092693 2025-10-22 15:00:16,292 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-10092693:ctest-vn-10092693 2025-10-22 15:00:16,864 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-61299989 2025-10-22 15:00:16,911 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-61299989:ctest-vn-61299989 2025-10-22 15:00:16,963 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyIpv4v6-39310351:ctest-vn-61299989:ctest-vn-61299989 2025-10-22 15:00:17,454 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:21,809 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:22,197 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:26,667 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-26312151:ctest-vn4-54091485:ctest-vn4-54091485 2025-10-22 15:00:26,715 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-26312151:ctest-vn4-54091485:ctest-vn4-54091485 2025-10-22 15:00:26,763 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-26312151:ctest-vn4-54091485:ctest-vn4-54091485 2025-10-22 15:00:27,531 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-26312151:ctest-vn4-54091485 2025-10-22 15:00:27,579 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-26312151:ctest-vn4-54091485:ctest-vn4-54091485 2025-10-22 15:00:27,632 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicyNegativeIpv4v6-26312151:ctest-vn4-54091485:ctest-vn4-54091485 2025-10-22 15:00:28,327 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:28,713 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:32,737 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:00:38,051 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153:ctest-vn-92186153 2025-10-22 15:00:38,103 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153:ctest-vn-92186153 2025-10-22 15:00:38,151 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153:ctest-vn-92186153 2025-10-22 15:00:39,580 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780:ctest-vn-82963780 2025-10-22 15:00:39,635 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780:ctest-vn-82963780 2025-10-22 15:00:39,692 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780:ctest-vn-82963780 2025-10-22 15:00:48,157 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:630d5268-85ee-47f1-b22a-f5e8e34022a5 2025-10-22 15:00:48,228 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153:ctest-vn-92186153 2025-10-22 15:00:48,279 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:630d5268-85ee-47f1-b22a-f5e8e34022a5 2025-10-22 15:00:59,912 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a1a65a44-a324-4aa9-bd04-9ae0c7c59491 2025-10-22 15:00:59,963 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780:ctest-vn-82963780 2025-10-22 15:01:00,015 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a1a65a44-a324-4aa9-bd04-9ae0c7c59491 2025-10-22 15:01:12,752 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780 2025-10-22 15:01:12,799 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780:ctest-vn-82963780 2025-10-22 15:01:12,847 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-82963780:ctest-vn-82963780 2025-10-22 15:01:13,420 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153 2025-10-22 15:01:13,471 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153:ctest-vn-92186153 2025-10-22 15:01:13,519 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicPolicy-70042290:ctest-vn-92186153:ctest-vn-92186153 2025-10-22 15:01:13,997 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:01:18,001 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:01:29,137 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:1ad37d35-fe82-4022-952c-de93c5d61756 2025-10-22 15:01:29,188 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRP-81114313:ctest-bgpaas_vn-59169274:ctest-bgpaas_vn-59169274 2025-10-22 15:01:29,239 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:1ad37d35-fe82-4022-952c-de93c5d61756 2025-10-22 15:01:40,812 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:147df6dd-113d-40ef-aa32-0e9f76562187 2025-10-22 15:01:40,863 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestRP-81114313:ctest-bgpaas_vn-50432022:ctest-bgpaas_vn-50432022 2025-10-22 15:01:40,915 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:147df6dd-113d-40ef-aa32-0e9f76562187 2025-10-22 15:01:42,743 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:01:49,988 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:01:54,021 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:01:57,411 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:01:57,459 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:01:57,507 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:01:59,603 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:01:59,655 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:01:59,703 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:02:01,891 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:02:01,943 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:02:01,991 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:02:19,260 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:5547bcac-79a1-4df2-8fcd-679f8e537ce9 2025-10-22 15:02:19,313 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:02:19,363 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ae6869e8-0f5a-4741-9597-68db0cfa1711 2025-10-22 15:02:19,415 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:02:19,467 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66917467-3b0c-4bea-be89-55ddc4b06370 2025-10-22 15:02:19,515 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:02:19,568 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:5547bcac-79a1-4df2-8fcd-679f8e537ce9 2025-10-22 15:02:19,631 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66917467-3b0c-4bea-be89-55ddc4b06370 2025-10-22 15:02:19,692 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ae6869e8-0f5a-4741-9597-68db0cfa1711 2025-10-22 15:02:58,956 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:c34a97dd-5abf-4095-ac7b-c2fa52c345ca 2025-10-22 15:02:59,007 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:02:59,059 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:c34a97dd-5abf-4095-ac7b-c2fa52c345ca 2025-10-22 15:03:01,288 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:588762b4-3363-4b60-bf1c-700fc28020c5 2025-10-22 15:03:01,339 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:03:01,387 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:588762b4-3363-4b60-bf1c-700fc28020c5 2025-10-22 15:03:07,949 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:5547bcac-79a1-4df2-8fcd-679f8e537ce9 2025-10-22 15:03:07,999 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66917467-3b0c-4bea-be89-55ddc4b06370 2025-10-22 15:03:08,048 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ae6869e8-0f5a-4741-9597-68db0cfa1711 2025-10-22 15:03:09,358 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981 2025-10-22 15:03:09,411 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ae6869e8-0f5a-4741-9597-68db0cfa1711 2025-10-22 15:03:09,463 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-22 15:03:09,478 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:03:09,686 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=2 2025-10-22 15:03:09,894 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Layer2RouteReq?x=2 2025-10-22 15:03:09,949 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981 2025-10-22 15:03:09,999 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:03:10,198 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=2 2025-10-22 15:03:10,418 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Layer2RouteReq?x=2 2025-10-22 15:03:10,433 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136 2025-10-22 15:03:10,483 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:5547bcac-79a1-4df2-8fcd-679f8e537ce9 2025-10-22 15:03:10,531 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-22 15:03:10,586 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:03:10,782 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=4 2025-10-22 15:03:10,998 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Layer2RouteReq?x=4 2025-10-22 15:03:11,011 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136 2025-10-22 15:03:11,059 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:03:11,258 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=4 2025-10-22 15:03:11,467 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Layer2RouteReq?x=4 2025-10-22 15:03:11,480 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723 2025-10-22 15:03:11,535 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66917467-3b0c-4bea-be89-55ddc4b06370 2025-10-22 15:03:11,587 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-22 15:03:11,600 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:03:11,810 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=3 2025-10-22 15:03:12,015 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Layer2RouteReq?x=3 2025-10-22 15:03:12,029 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723 2025-10-22 15:03:12,079 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:03:12,286 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=3 2025-10-22 15:03:12,495 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Layer2RouteReq?x=3 2025-10-22 15:03:13,222 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name= 2025-10-22 15:03:13,276 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=5 2025-10-22 15:03:13,333 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:ae6869e8-0f5a-4741-9597-68db0cfa1711 2025-10-22 15:03:13,490 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name= 2025-10-22 15:03:13,540 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_Inet4UcRouteReq?x=6 2025-10-22 15:03:13,594 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:66917467-3b0c-4bea-be89-55ddc4b06370 2025-10-22 15:03:24,060 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723 2025-10-22 15:03:24,111 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:03:24,163 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-right_vn-59837723:ctest-right_vn-59837723 2025-10-22 15:03:24,755 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981 2025-10-22 15:03:24,807 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:03:24,855 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-left_vn-97397981:ctest-left_vn-97397981 2025-10-22 15:03:25,508 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136 2025-10-22 15:03:25,559 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:03:25,609 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestSvcRegr-77372160:ctest-mgmt_vn-29952136:ctest-mgmt_vn-29952136 2025-10-22 15:03:26,109 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:03:30,584 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:03:40,616 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7d190147-6e20-490c-a535-28e5f7460b8c 2025-10-22 15:03:40,667 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestvDNSBasic0-66791653:ctest-vn1-vdns-33110776:ctest-vn1-vdns-33110776 2025-10-22 15:03:40,723 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7d190147-6e20-490c-a535-28e5f7460b8c 2025-10-22 15:03:54,930 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_DnsInfo? 2025-10-22 15:04:02,192 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:f7f030d4-cd4b-4ca3-8a4e-bb5357532898 2025-10-22 15:04:02,244 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestvDNSBasic0-66791653:ctest-vn1-vdns-33110776:ctest-vn1-vdns-33110776 2025-10-22 15:04:02,295 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:f7f030d4-cd4b-4ca3-8a4e-bb5357532898 2025-10-22 15:04:20,009 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_DnsInfo? 2025-10-22 15:04:32,157 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:04:36,273 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:04:46,304 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:fb0fe844-c692-4de2-94d0-47ae7454bc11 2025-10-22 15:04:46,357 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVNx-77907778:ctest-vn-61545137:ctest-vn-61545137 2025-10-22 15:04:46,411 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:fb0fe844-c692-4de2-94d0-47ae7454bc11 2025-10-22 15:04:58,008 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:393a1343-5caf-446f-8d71-f6bc44c02a7b 2025-10-22 15:04:58,063 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVNx-77907778:ctest-vn-61545137:ctest-vn-61545137 2025-10-22 15:04:58,115 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:393a1343-5caf-446f-8d71-f6bc44c02a7b 2025-10-22 15:05:06,857 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:05:10,949 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:05:19,454 - DEBUG - [10.0.0.241]: Running cmd : docker exec --privileged -it vrouter-vrouter-agent-1 /bin/bash -c 'echo "10.0.0.241 cn-jenkins-deploy-platform-juju-os-601-1-test" >> /etc/hosts' 2025-10-22 15:05:19,770 - DEBUG - Output : 2025-10-22 15:05:21,207 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:3e8cb167-6a29-4d35-855f-fb5155227c13 2025-10-22 15:05:21,269 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn2_metadata-13007884:ctest-vn2_metadata-13007884 2025-10-22 15:05:21,347 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:3e8cb167-6a29-4d35-855f-fb5155227c13 2025-10-22 15:05:33,660 - DEBUG - [10.0.0.241]: Running cmd : docker exec --privileged -it vrouter-vrouter-agent-1 /bin/bash -c 'sed -i '$ d' /etc/hosts' 2025-10-22 15:05:33,869 - DEBUG - Output : sed: -e expression #1, char 1: missing command 2025-10-22 15:05:36,801 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:05:37,197 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:05:45,064 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9ccbbe3c-9376-458b-b8aa-ddc93603702d 2025-10-22 15:05:45,115 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-98116839:ctest-vn-98116839 2025-10-22 15:05:45,167 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9ccbbe3c-9376-458b-b8aa-ddc93603702d 2025-10-22 15:05:57,440 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:05:57,848 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:06:05,560 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-27319403:ctest-vn-27319403 2025-10-22 15:06:06,180 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:96cd60c7-fc3d-4ffa-8a6a-e5c4d4a8f61a 2025-10-22 15:06:06,236 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-27319403:ctest-vn-27319403 2025-10-22 15:06:06,322 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:96cd60c7-fc3d-4ffa-8a6a-e5c4d4a8f61a 2025-10-22 15:06:16,968 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:af739acc-1d6e-4f13-a331-a7de763b7bf2 2025-10-22 15:06:17,020 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-27319403:ctest-vn-27319403 2025-10-22 15:06:17,071 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:af739acc-1d6e-4f13-a331-a7de763b7bf2 2025-10-22 15:06:19,285 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d61c8211-898b-45d2-9915-ac7d585a1544 2025-10-22 15:06:19,335 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-27319403:ctest-vn-27319403 2025-10-22 15:06:19,383 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d61c8211-898b-45d2-9915-ac7d585a1544 2025-10-22 15:06:38,188 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VnListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-27319403 2025-10-22 15:06:38,235 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name=admin_domain:ctest-TestBasicVMVN-36841997:ctest-vn-27319403:ctest-vn-27319403 2025-10-22 15:06:38,284 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_VrfListReq?name= 2025-10-22 15:06:38,336 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_KRouteReq?x=2 2025-10-22 15:06:38,391 - DEBUG - Unable to fetch route details in vrouter for vrf index 2, Got :[] 2025-10-22 15:06:39,029 - DEBUG - Requesting: https://10.0.0.241:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-22 15:08:47,176 - INFO - Using existing project ['admin_domain', 'admin'](9de6d401-db5c-425c-b65a-d62f9ed1f01c) 2025-10-22 15:08:47,186 - INFO - ================================================================================ 2025-10-22 15:08:47,187 - INFO - STARTING TEST : test_contrail_status 2025-10-22 15:08:47,187 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2025-10-22 15:08:47,406 - DEBUG - Skipping xmpp flap check 2025-10-22 15:08:47,406 - INFO - Initial checks done. Running the testcase now 2025-10-22 15:08:47,406 - INFO - 2025-10-22 15:08:47,531 - INFO - 10.0.0.241 2025-10-22 15:08:47,531 - INFO - config 2025-10-22 15:08:47,588 - INFO - config-nodemgr:active 2025-10-22 15:08:47,643 - INFO - api-server:active 2025-10-22 15:08:47,699 - INFO - schema:active 2025-10-22 15:08:47,751 - INFO - svc-monitor:active 2025-10-22 15:08:47,811 - INFO - device-manager:active 2025-10-22 15:08:47,811 - INFO - config-database 2025-10-22 15:08:47,811 - INFO - config-cassandra:active 2025-10-22 15:08:47,811 - INFO - config-zookeeper:active 2025-10-22 15:08:47,811 - INFO - config-rabbitmq:active 2025-10-22 15:08:47,811 - INFO - vrouter 2025-10-22 15:08:47,867 - INFO - vrouter-nodemgr:active 2025-10-22 15:08:47,923 - INFO - agent:active 2025-10-22 15:08:47,923 - INFO - control 2025-10-22 15:08:47,983 - INFO - control-nodemgr:active 2025-10-22 15:08:48,051 - INFO - control:active 2025-10-22 15:08:48,051 - INFO - named:active 2025-10-22 15:08:48,107 - INFO - dns:active 2025-10-22 15:08:48,107 - INFO - analytics 2025-10-22 15:08:48,163 - INFO - analytics-nodemgr:active 2025-10-22 15:08:48,215 - INFO - analytics-api:active 2025-10-22 15:08:48,275 - INFO - collector:active 2025-10-22 15:08:48,275 - INFO - analytics-database 2025-10-22 15:08:48,275 - INFO - analytics-cassandra:active 2025-10-22 15:08:48,331 - INFO - analyticsdb-nodemgr:active 2025-10-22 15:08:48,387 - INFO - query-engine:active 2025-10-22 15:08:48,387 - INFO - webui 2025-10-22 15:08:48,387 - INFO - webui:active 2025-10-22 15:08:48,387 - INFO - webui-middleware:active 2025-10-22 15:08:48,387 - INFO - redis:active 2025-10-22 15:08:48,387 - INFO - analytics_snmp 2025-10-22 15:08:48,439 - INFO - snmp-collector:active 2025-10-22 15:08:48,439 - INFO - snmp-topology:active 2025-10-22 15:08:48,439 - INFO - analytics_alarm 2025-10-22 15:08:48,439 - INFO - alarmgen:active 2025-10-22 15:08:48,439 - INFO - All the contrail services are up on all nodes 2025-10-22 15:08:48,439 - INFO - contrail-status passed 2025-10-22 15:08:48,654 - DEBUG - Skipping xmpp flap check
2025-10-22 15:08:48,654 - INFO - END TEST : test_contrail_status : PASSED[0:00:01]
2025-10-22 15:08:48,654 - INFO - --------------------------------------------------------------------------------