2025-07-24 05:47:34,338 - INFO - Domain Default found not creating 2025-07-24 05:47:34,531 - INFO - Project ctest-TestBasicVMVN0-28589761 not found, creating it 2025-07-24 05:47:35,291 - INFO - Created Project:ctest-TestBasicVMVN0-28589761, ID : d2e93d5c-b4fb-4404-9fa1-32cb331bdd8a 2025-07-24 05:47:37,167 - INFO - ================================================================================ 2025-07-24 05:47:37,167 - INFO - STARTING TEST : test_control_node_switchover 2025-07-24 05:47:37,168 - INFO - TEST DESCRIPTION : Stop the control node and check peering with agent fallback to other control node. 1. Pick one VN from respource pool which has 2 VM's in it 2. Verify ping between VM's 3. Find active control node in cluster by agent inspect 4. Stop control service on active control node 5. Verify agents are connected to new active control-node using xmpp connections 6. Bring back control service on previous active node 7. Verify ping between VM's again after bringing up control serveice Pass criteria: Step 2,5 and 7 should pass 2025-07-24 05:47:38,468 - DEBUG - Skipping xmpp flap check 2025-07-24 05:47:38,468 - INFO - Initial checks done. Running the testcase now 2025-07-24 05:47:38,468 - INFO - 2025-07-24 05:47:39,275 - DEBUG - Response for create_network : {'network': {'id': '9f0fa338-d809-4542-bbc0-609191c140e5', 'name': 'ctest-vn1-23662782', 'tenant_id': 'd2e93d5cb4fb44049fa132cb331bdd8a', 'project_id': 'd2e93d5cb4fb44049fa132cb331bdd8a', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBasicVMVN0-28589761', 'ctest-vn1-23662782'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 05:47:39,637 - DEBUG - Response for create_subnet : {'subnet': {'id': '6b9fec6d-49d5-4a6f-9501-7f76cb108295', 'name': '', 'tenant_id': 'd2e93d5cb4fb44049fa132cb331bdd8a', 'network_id': '9f0fa338-d809-4542-bbc0-609191c140e5', 'ip_version': 4, 'cidr': '192.168.1.0/24', 'allocation_pools': [{'start': '192.168.1.2', 'end': '192.168.1.254'}], 'gateway_ip': '192.168.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '192.168.1.2', 'tags': [], 'project_id': 'd2e93d5cb4fb44049fa132cb331bdd8a'}} 2025-07-24 05:47:39,662 - INFO - Created VN ctest-vn1-23662782 2025-07-24 05:47:39,682 - DEBUG - VN ctest-vn1-23662782 UUID is 9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:39,784 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 05:47:40,127 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 05:47:40,162 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 05:47:40,184 - DEBUG - Requesting: http://10.0.0.19:8082/project/d2e93d5c-b4fb-4404-9fa1-32cb331bdd8a 2025-07-24 05:47:40,318 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:40,360 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:40,373 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/9a6c04c6-d62b-45c1-9600-ef150949c278 2025-07-24 05:47:40,384 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/9a6c04c6-d62b-45c1-9600-ef150949c278 2025-07-24 05:47:40,396 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/1ce4dc35-4530-4d74-8d3d-4a644c643147 2025-07-24 05:47:40,411 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:47:40,411 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:40,425 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/9a6c04c6-d62b-45c1-9600-ef150949c278 2025-07-24 05:47:40,451 - INFO - Verified VN network id 13 for VN 9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:40,451 - INFO - Verifications in API Server for VN ctest-vn1-23662782 passed 2025-07-24 05:47:40,451 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:40,482 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/9a6c04c6-d62b-45c1-9600-ef150949c278 2025-07-24 05:47:40,497 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/9a6c04c6-d62b-45c1-9600-ef150949c278 2025-07-24 05:47:40,513 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/1ce4dc35-4530-4d74-8d3d-4a644c643147 2025-07-24 05:47:40,533 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-28589761', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11461559041219183938', 'uuid-lslong': '13528919458952855781'}, 'enable': 'true', 'created': '2025-07-24T05:47:39', 'last-modified': '2025-07-24T05:47:39', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.989697'} 2025-07-24 05:47:40,543 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:47:40,550 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-28589761', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11461559041219183938', 'uuid-lslong': '13528919458952855781'}, 'enable': 'true', 'created': '2025-07-24T05:47:39', 'last-modified': '2025-07-24T05:47:39', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.014363'} 2025-07-24 05:47:40,558 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:47:40,567 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-28589761', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11461559041219183938', 'uuid-lslong': '13528919458952855781'}, 'enable': 'true', 'created': '2025-07-24T05:47:39', 'last-modified': '2025-07-24T05:47:39', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.032064'} 2025-07-24 05:47:40,577 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:47:40,578 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-23662782 passed 2025-07-24 05:47:40,578 - DEBUG - ====Verifying policy data for ctest-vn1-23662782 in API_Server ====== 2025-07-24 05:47:40,578 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 05:47:40,587 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 05:47:40,602 - DEBUG - Requesting: http://10.0.0.19:8082/project/d2e93d5c-b4fb-4404-9fa1-32cb331bdd8a 2025-07-24 05:47:40,716 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/9f0fa338-d809-4542-bbc0-609191c140e5 2025-07-24 05:47:40,729 - DEBUG - =>VN ctest-vn1-23662782 has no policy to be verified 2025-07-24 05:47:40,729 - DEBUG - Verifying the vn in opserver 2025-07-24 05:47:40,729 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 virtual network link through opserver 10.0.0.19 2025-07-24 05:47:40,729 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 05:47:41,065 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 05:47:41,163 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782?flat'} 2025-07-24 05:47:41,163 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 virtual network link through opserver 10.0.0.38 2025-07-24 05:47:41,163 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 05:47:41,496 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 05:47:41,578 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782?flat'} 2025-07-24 05:47:41,578 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 virtual network link through opserver 10.0.0.41 2025-07-24 05:47:41,578 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 05:47:41,917 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 05:47:41,993 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782?flat'} 2025-07-24 05:47:41,993 - INFO - Validated that VN default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is found in opserver 2025-07-24 05:47:42,026 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 05:47:42,045 - DEBUG - VRF ids for VN ctest-vn1-23662782: {} 2025-07-24 05:47:42,231 - DEBUG - Services list from nova: [, , , ] 2025-07-24 05:47:43,619 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3712-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3712-1) 2025-07-24 05:47:45,314 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3712-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3712-2) 2025-07-24 05:47:45,314 - INFO - Waiting for VM ctest-vn1_vm1-27734985 to be up.. 2025-07-24 05:47:45,429 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-07-24 05:47:50,538 - DEBUG - VM is in ACTIVE state now 2025-07-24 05:47:50,538 - INFO - VM name : ctest-vn1_vm1-27734985 2025-07-24 05:47:50,636 - DEBUG - VM ctest-vn1_vm1-27734985 ID is bd644d76-ba09-4f7c-963d-fae05e4ee3fd 2025-07-24 05:47:50,671 - DEBUG - VM ctest-vn1_vm1-27734985 launched on Node an-jenkins-deploy-platform-ansible-os-3712-1 2025-07-24 05:47:50,782 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-machine/bd644d76-ba09-4f7c-963d-fae05e4ee3fd 2025-07-24 05:47:51,103 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-machine/bd644d76-ba09-4f7c-963d-fae05e4ee3fd 2025-07-24 05:47:51,156 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-machine-interface/8b76db37-347e-47e5-9d1b-e04f1110c512 2025-07-24 05:47:54,379 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1002ms') 2025-07-24 05:47:54,379 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:47:54,401 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:47:58,467 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-07-24 05:47:58,467 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:47:58,488 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:02,571 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-07-24 05:48:02,571 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:02,608 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:06,698 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1025ms') 2025-07-24 05:48:06,699 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:06,725 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:10,824 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1033ms') 2025-07-24 05:48:10,824 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:10,846 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:14,924 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-07-24 05:48:14,924 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:14,945 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:19,018 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-07-24 05:48:19,018 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:19,042 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:23,146 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1030ms') 2025-07-24 05:48:23,146 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:23,168 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:27,242 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1003ms') 2025-07-24 05:48:27,243 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:27,265 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:31,369 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1030ms') 2025-07-24 05:48:31,369 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 failed! 2025-07-24 05:48:31,392 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-23662782 is 192.168.1.1 and allocation pool is NOT set 2025-07-24 05:48:33,458 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=7.14 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.19 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 3.192/5.165/7.138/1.973 ms') 2025-07-24 05:48:33,458 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-27734985 passed 2025-07-24 05:48:33,539 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:48:33,539 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-27734985, IP 192.168.1.3, Port 22 2025-07-24 05:48:33,607 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-24 05:48:33,716 - DEBUG - VM ctest-vn1_vm1-27734985 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-24 05:48:38,717 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:48:38,717 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-27734985, IP 192.168.1.3, Port 22 2025-07-24 05:48:38,787 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-24 05:48:38,895 - DEBUG - VM ctest-vn1_vm1-27734985 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-24 05:48:43,896 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:48:43,897 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-27734985, IP 192.168.1.3, Port 22 2025-07-24 05:48:43,967 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-24 05:48:44,094 - DEBUG - VM ctest-vn1_vm1-27734985 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-24 05:48:49,095 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:48:49,095 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-27734985, IP 192.168.1.3, Port 22 2025-07-24 05:48:49,167 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-24 05:48:49,291 - DEBUG - VM ctest-vn1_vm1-27734985 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-24 05:48:54,292 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:48:54,292 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-27734985, IP 192.168.1.3, Port 22 2025-07-24 05:48:54,359 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-24 05:48:54,491 - DEBUG - VM ctest-vn1_vm1-27734985 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-24 05:48:59,492 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:48:59,492 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-27734985, IP 192.168.1.3, Port 22 2025-07-24 05:48:59,673 - DEBUG - VM ctest-vn1_vm1-27734985 is ready for SSH connections 2025-07-24 05:48:59,674 - INFO - Waiting for VM ctest-vn1_vm2-17184754 to be up.. 2025-07-24 05:48:59,792 - DEBUG - VM is in ACTIVE state now 2025-07-24 05:48:59,792 - INFO - VM name : ctest-vn1_vm2-17184754 2025-07-24 05:48:59,925 - DEBUG - VM ctest-vn1_vm2-17184754 ID is 5ccc5ca3-9bfd-4471-b61b-a13e0ad58112 2025-07-24 05:48:59,925 - DEBUG - VM ctest-vn1_vm2-17184754 launched on Node an-jenkins-deploy-platform-ansible-os-3712-2 2025-07-24 05:49:00,050 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-machine/5ccc5ca3-9bfd-4471-b61b-a13e0ad58112 2025-07-24 05:49:00,070 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-machine-interface/e58ec8b9-fb67-42eb-87b0-27443ee10331 2025-07-24 05:49:01,276 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.31 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.764 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.764/3.035/5.306/2.271 ms') 2025-07-24 05:49:01,276 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm2-17184754 passed 2025-07-24 05:49:01,369 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:49:01,369 - DEBUG - Waiting to SSH to VM ctest-vn1_vm2-17184754, IP 192.168.1.4, Port 22 2025-07-24 05:49:01,546 - DEBUG - VM ctest-vn1_vm2-17184754 is ready for SSH connections 2025-07-24 05:49:01,546 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:49:01,546 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 192.168.1.4, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-07-24 05:49:01,546 - DEBUG - ping -s 56 -c 3 -W 1 192.168.1.4 2025-07-24 05:49:29,254 - DEBUG - PING 192.168.1.4 (192.168.1.4) 56(84) bytes of data. 64 bytes from 192.168.1.4: icmp_req=1 ttl=64 time=11.5 ms 64 bytes from 192.168.1.4: icmp_req=2 ttl=64 time=1.16 ms 64 bytes from 192.168.1.4: icmp_req=3 ttl=64 time=1.25 ms --- 192.168.1.4 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 1.160/4.667/11.590/4.895 ms 2025-07-24 05:49:29,254 - INFO - Ping to IP 192.168.1.4 from VM ctest-vn1_vm1-27734985 passed 2025-07-24 05:49:29,254 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-24 05:49:29,254 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 192.168.1.3, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.23, gateway password: c0ntrail123 2025-07-24 05:49:29,254 - DEBUG - ping -s 56 -c 3 -W 1 192.168.1.3 2025-07-24 05:50:17,152 - DEBUG - PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data. 64 bytes from 192.168.1.3: icmp_req=1 ttl=64 time=7.88 ms 64 bytes from 192.168.1.3: icmp_req=2 ttl=64 time=2.30 ms 64 bytes from 192.168.1.3: icmp_req=3 ttl=64 time=1.76 ms --- 192.168.1.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.763/3.983/7.880/2.764 ms 2025-07-24 05:50:17,152 - INFO - Ping to IP 192.168.1.3 from VM ctest-vn1_vm2-17184754 passed 2025-07-24 05:50:17,165 - INFO - Active control node from the Agent 10.0.0.57 is 10.0.0.41 2025-07-24 05:50:17,165 - INFO - Stoping the Control service in 10.0.0.41 2025-07-24 05:50:17,660 - INFO - 10.0.0.41 2025-07-24 05:50:17,660 - INFO - control:inactive 2025-07-24 05:50:17,660 - INFO - Contrail services control are down on nodes 10.0.0.41 2025-07-24 05:50:17,664 - INFO - Deleting VM ctest-vn1_vm2-17184754 2025-07-24 05:50:17,746 - INFO - Deleting VM ctest-vn1_vm1-27734985 2025-07-24 05:50:17,840 - INFO - Deleting VN ctest-vn1-23662782 2025-07-24 05:50:17,896 - DEBUG - VN 9f0fa338-d809-4542-bbc0-609191c140e5 still in use: Unable to complete operation on network 9f0fa338-d809-4542-bbc0-609191c140e5. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-efff6035-2eec-4323-8c99-d77f92e2a6dd'] 2025-07-24 05:50:17,896 - WARNING - Deleting VN ctest-vn1-23662782 failed..Will retry 2025-07-24 05:50:20,177 - DEBUG - Response for deleting network () 2025-07-24 05:50:20,177 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/9a6c04c6-d62b-45c1-9600-ef150949c278 2025-07-24 05:50:20,186 - DEBUG - Response Code: 404 2025-07-24 05:50:20,187 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 05:50:20,195 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 05:50:20,277 - DEBUG - Requesting: http://10.0.0.19:8082/project/d2e93d5c-b4fb-4404-9fa1-32cb331bdd8a 2025-07-24 05:50:20,389 - INFO - Validated that VN ctest-vn1-23662782 is not found in API Server 2025-07-24 05:50:20,407 - DEBUG - VN ctest-vn1-23662782 is not present in Agent 10.0.0.57 2025-07-24 05:50:20,421 - DEBUG - VN ctest-vn1-23662782 is not present in Agent 10.0.0.23 2025-07-24 05:50:20,421 - INFO - Validated that VN ctest-vn1-23662782 is not in any agent 2025-07-24 05:50:21,780 - ERROR - NameError Python 3.6.8: /usr/bin/python3 Thu Jul 24 05:50:17 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_control_node_switchover(self=) 652 'contrail-control', [active_controller_host_ip], 653 container='control') 654 sleep(5) 655 656 # Check the control node shifted to other control node sleep undefined NameError: name 'sleep' is not defined __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Name not found globally.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("name 'sleep' is not defined",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 654, in test_control_node_switchover sleep(5) NameError: name 'sleep' is not defined 2025-07-24 05:50:21,781 - DEBUG - Skipping xmpp flap check 2025-07-24 05:50:21,781 - INFO - 2025-07-24 05:50:21,781 - INFO - END TEST : test_control_node_switchover : FAILED[0:02:44] 2025-07-24 05:50:21,781 - INFO - -------------------------------------------------------------------------------- 2025-07-24 05:50:21,787 - INFO - ================================================================================ 2025-07-24 05:50:21,787 - INFO - STARTING TEST : test_process_restart_in_policy_between_vns 2025-07-24 05:50:21,787 - INFO - TEST DESCRIPTION : Test to validate that with policy having rule to check icmp fwding between VMs on different VNs , ping between VMs should pass with process restarts 1. Pick 2 VN's from resource pool which has one VM each 2. Create policy with icmp allow rule between those VN's and bind it networks 3. Ping from one VM to another VM 4. Restart process 'vrouter' and 'control' on setup 5. Ping again between VM's after process restart Pass criteria: Step 2,3,4 and 5 should pass 2025-07-24 05:50:23,104 - DEBUG - Skipping xmpp flap check 2025-07-24 05:50:23,104 - INFO - Initial checks done. Running the testcase now 2025-07-24 05:50:23,104 - INFO - 2025-07-24 05:50:23,113 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-03203116, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn2-80310121, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-24 05:50:23,468 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn2-80310121, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-03203116, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-24 05:50:24,112 - INFO - Created VN ctest-vn1-03203116, UUID :a4d93401-d362-44c2-bca2-541834fd0516 2025-07-24 05:50:24,567 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 05:50:24,577 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 05:50:24,607 - DEBUG - Requesting: http://10.0.0.19:8082/project/d2e93d5c-b4fb-4404-9fa1-32cb331bdd8a 2025-07-24 05:50:24,741 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a4d93401-d362-44c2-bca2-541834fd0516 2025-07-24 05:50:24,763 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a4d93401-d362-44c2-bca2-541834fd0516 2025-07-24 05:50:24,782 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/65e7e264-a1dd-4a0b-84b3-abac3539fb6a 2025-07-24 05:50:24,795 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/65e7e264-a1dd-4a0b-84b3-abac3539fb6a 2025-07-24 05:50:24,807 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/32c3f71f-4c66-45df-8e0a-81fd6d951f4b 2025-07-24 05:50:24,819 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:50:24,819 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a4d93401-d362-44c2-bca2-541834fd0516 2025-07-24 05:50:24,850 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/65e7e264-a1dd-4a0b-84b3-abac3539fb6a 2025-07-24 05:50:24,889 - INFO - Verified VN network id 13 for VN a4d93401-d362-44c2-bca2-541834fd0516 2025-07-24 05:50:24,889 - INFO - Verifications in API Server for VN ctest-vn1-03203116 passed 2025-07-24 05:50:24,889 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a4d93401-d362-44c2-bca2-541834fd0516 2025-07-24 05:50:24,912 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/65e7e264-a1dd-4a0b-84b3-abac3539fb6a 2025-07-24 05:50:24,927 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/65e7e264-a1dd-4a0b-84b3-abac3539fb6a 2025-07-24 05:50:24,937 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/32c3f71f-4c66-45df-8e0a-81fd6d951f4b 2025-07-24 05:50:24,961 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-03203116', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-28589761', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11878582674612438210', 'uuid-lslong': '13592519088302523670'}, 'enable': 'true', 'created': '2025-07-24T05:50:24', 'last-modified': '2025-07-24T05:50:24', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.670772'} 2025-07-24 05:50:24,976 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:50:24,986 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-03203116', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-28589761', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11878582674612438210', 'uuid-lslong': '13592519088302523670'}, 'enable': 'true', 'created': '2025-07-24T05:50:24', 'last-modified': '2025-07-24T05:50:24', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.694517'} 2025-07-24 05:50:25,003 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 05:50:25,009 - INFO - Deleting VN ctest-vn1-03203116 2025-07-24 05:50:25,275 - INFO - Deleted policy policy2 2025-07-24 05:50:25,333 - INFO - Deleted policy policy1 2025-07-24 05:50:26,643 - ERROR - AttributeError Python 3.6.8: /usr/bin/python3 Thu Jul 24 05:50:25 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_process_restart_in_policy_between_vns(self=) 397 connections=self.connections)) 398 vn1_fixture = self.create_vn(vn1_name, vn1_subnets, option='contrail') 399 assert vn1_fixture.verify_on_setup() 400 vn1_fixture.bind_policies( 401 [policy1_fixture.policy_fq_name], vn1_fixture.vn_id) vn1_fixture = vn1_fixture.verify_on_setup = > /contrail-test/fixtures/vn_test.py in verify_on_setup(self=) 579 "One or more verifications in API Server for VN %s failed" % (self.vn_name)) 580 return result 581 if not self.verify_vn_in_control_nodes(): 582 result = result and False 583 self.logger.error( self = self.verify_vn_in_control_nodes = .deco_retry..f_retry of > /contrail-test/tcutils/util.py in f_retry(*args=(,), **kwargs={}) 82 mtries, mdelay = tries, delay # make mutable 83 84 result = f(*args, **kwargs) # first attempt 85 rv = result 86 final = False result undefined f = args = (,) kwargs = {} /contrail-test/fixtures/vn_test.py in verify_vn_in_control_nodes(self=) 927 for cn in self.inputs.bgp_ips: 928 cn_config_vn_obj = self.cn_inspect[cn].get_cn_config_vn( 929 vn_name=self.vn_name, project=self.project_name, domain=self.domain_name) 930 if not cn_config_vn_obj: 931 self.logger.warn('Control-node %s does not have VN %s info ' % vn_name undefined self = self.vn_name = 'ctest-vn1-03203116' project undefined self.project_name = 'ctest-TestBasicVMVN0-28589761' domain undefined self.domain_name = 'default-domain' /contrail-test/tcutils/control/cn_introspect_utils.py in get_cn_config_vn(self=, domain='default-domain', project='ctest-TestBasicVMVN0-28589761', vn_name='ctest-vn1-03203116') 139 def get_cn_config_vn(self, domain='default-domain', project='admin', vn_name='default-virtual-network'): 140 m = 'virtual-network:' + domain + ':' + project + ':' + vn_name 141 return self._get_if_map_table_entry(m) 142 143 def get_cn_config_fip_pool(self, domain='default-domain', project='admin', vn_name='default-virtual-network', fip_pool_name='default-floating-ip-pool'): self = self._get_if_map_table_entry = > m = 'virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-03203116' /contrail-test/tcutils/control/cn_introspect_utils.py in _get_if_map_table_entry(self=, match='virtual-network:default-domain:ctest-TestBasicVMVN0-28589761:ctest-vn1-03203116') 37 new_table_req = 'Snh_IFMapTableShowReq?table_name=' + table_name.group(1) + '&search_string=' + match 38 p = self.dict_get(new_table_req) 39 xp = p.xpath('./IFMapTableShowResp/ifmap_db/list/IFMapNodeShowInfo') 40 if not xp: 41 # sometime ./xpath dosen't work; work around xp undefined p = None p.xpath undefined AttributeError: 'NoneType' object has no attribute 'xpath' __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Attribute not found.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("'NoneType' object has no attribute 'xpath'",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 399, in test_process_restart_in_policy_between_vns assert vn1_fixture.verify_on_setup() File "/contrail-test/fixtures/vn_test.py", line 581, in verify_on_setup if not self.verify_vn_in_control_nodes(): File "/contrail-test/tcutils/util.py", line 84, in f_retry result = f(*args, **kwargs) # first attempt File "/contrail-test/fixtures/vn_test.py", line 929, in verify_vn_in_control_nodes vn_name=self.vn_name, project=self.project_name, domain=self.domain_name) File "/contrail-test/tcutils/control/cn_introspect_utils.py", line 141, in get_cn_config_vn return self._get_if_map_table_entry(m) File "/contrail-test/tcutils/control/cn_introspect_utils.py", line 39, in _get_if_map_table_entry xp = p.xpath('./IFMapTableShowResp/ifmap_db/list/IFMapNodeShowInfo') AttributeError: 'NoneType' object has no attribute 'xpath' 2025-07-24 05:50:26,644 - DEBUG - Skipping xmpp flap check 2025-07-24 05:50:26,644 - INFO - 2025-07-24 05:50:26,644 - INFO - END TEST : test_process_restart_in_policy_between_vns : FAILED[0:00:05] 2025-07-24 05:50:26,644 - INFO - -------------------------------------------------------------------------------- 2025-07-24 05:50:26,650 - INFO - ================================================================================ 2025-07-24 05:50:26,650 - INFO - STARTING TEST : test_underlay_broadcast_traffic_handling 2025-07-24 05:50:26,650 - INFO - TEST DESCRIPTION : Test the underlay brocast traffic handling by vrouter. (Bug-1545229). 1. Send broadcast traffic from one compute node. 2. Other compute in same subnet should receive that traffic. 3. Receiving compute should treat this traffic as underlay. 4. Compute should not replicate the packet and send the copy back. Pass criteria: Step 3-4 should pass Maintainer : chhandak@juniper.net 2025-07-24 05:50:27,963 - DEBUG - Skipping xmpp flap check 2025-07-24 05:50:27,963 - INFO - Initial checks done. Running the testcase now 2025-07-24 05:50:27,963 - INFO - 2025-07-24 05:50:28,349 - INFO - Agent interface name: ens6 2025-07-24 05:50:28,423 - DEBUG - Executing command: sudo tcpdump -nni ens6 -U host 10.20.0.255 -w /tmp/ens6_ctest-random-16582708.pcap 2025-07-24 05:50:29,834 - ERROR - NameError Python 3.6.8: /usr/bin/python3 Thu Jul 24 05:50:28 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_underlay_broadcast_traffic_handling(self=) 914 compute_password, comp_intf, filters, self.logger) 915 916 sleep(5) 917 918 # Ping broadcast address sleep undefined NameError: name 'sleep' is not defined __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Name not found globally.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("name 'sleep' is not defined",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 916, in test_underlay_broadcast_traffic_handling sleep(5) NameError: name 'sleep' is not defined 2025-07-24 05:50:29,835 - DEBUG - Skipping xmpp flap check 2025-07-24 05:50:29,835 - INFO - 2025-07-24 05:50:29,835 - INFO - END TEST : test_underlay_broadcast_traffic_handling : FAILED[0:00:03] 2025-07-24 05:50:29,835 - INFO - -------------------------------------------------------------------------------- 2025-07-24 05:50:30,729 - INFO - Deleted project: ctest-TestBasicVMVN0-28589761, ID : d2e93d5c-b4fb-4404-9fa1-32cb331bdd8a