2025-07-22 18:16:26,224 - INFO - Domain Default found not creating 2025-07-22 18:16:26,387 - INFO - Project ctest-TestBasicVMVN0-42431603 not found, creating it 2025-07-22 18:16:27,018 - INFO - Created Project:ctest-TestBasicVMVN0-42431603, ID : f557f3c8-3611-41ae-9e19-02699aa8ef0a 2025-07-22 18:16:29,463 - INFO - ================================================================================ 2025-07-22 18:16:29,463 - INFO - STARTING TEST : test_control_node_switchover 2025-07-22 18:16:29,466 - INFO - TEST DESCRIPTION : Stop the control node and check peering with agent fallback to other control node. 1. Pick one VN from respource pool which has 2 VM's in it 2. Verify ping between VM's 3. Find active control node in cluster by agent inspect 4. Stop control service on active control node 5. Verify agents are connected to new active control-node using xmpp connections 6. Bring back control service on previous active node 7. Verify ping between VM's again after bringing up control serveice Pass criteria: Step 2,5 and 7 should pass 2025-07-22 18:16:30,737 - DEBUG - Skipping xmpp flap check 2025-07-22 18:16:30,737 - INFO - Initial checks done. Running the testcase now 2025-07-22 18:16:30,738 - INFO - 2025-07-22 18:16:31,462 - DEBUG - Response for create_network : {'network': {'id': 'a851bfb3-788f-4ec3-9004-b50642de31e4', 'name': 'ctest-vn1-30521313', 'tenant_id': 'f557f3c8361141ae9e1902699aa8ef0a', 'project_id': 'f557f3c8361141ae9e1902699aa8ef0a', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBasicVMVN0-42431603', 'ctest-vn1-30521313'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 18:16:31,725 - DEBUG - Response for create_subnet : {'subnet': {'id': 'f3ff0cf4-1c22-4cd4-b954-8ca71d5266d0', 'name': '', 'tenant_id': 'f557f3c8361141ae9e1902699aa8ef0a', 'network_id': 'a851bfb3-788f-4ec3-9004-b50642de31e4', 'ip_version': 4, 'cidr': '192.168.1.0/24', 'allocation_pools': [{'start': '192.168.1.2', 'end': '192.168.1.254'}], 'gateway_ip': '192.168.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '192.168.1.2', 'tags': [], 'project_id': 'f557f3c8361141ae9e1902699aa8ef0a'}} 2025-07-22 18:16:31,752 - INFO - Created VN ctest-vn1-30521313 2025-07-22 18:16:31,769 - DEBUG - VN ctest-vn1-30521313 UUID is a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:31,843 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 18:16:32,136 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 18:16:32,170 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 18:16:32,191 - DEBUG - Requesting: http://10.0.0.254:8082/project/f557f3c8-3611-41ae-9e19-02699aa8ef0a 2025-07-22 18:16:32,298 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:32,312 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:32,322 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/0271c8f9-85eb-423b-a83d-4fd53f1fd5c7 2025-07-22 18:16:32,331 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/0271c8f9-85eb-423b-a83d-4fd53f1fd5c7 2025-07-22 18:16:32,341 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/9c7748c6-3e4b-4d98-8d39-929b1e6ef93d 2025-07-22 18:16:32,350 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:16:32,350 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:32,361 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/0271c8f9-85eb-423b-a83d-4fd53f1fd5c7 2025-07-22 18:16:32,382 - INFO - Verified VN network id 12 for VN a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:32,382 - INFO - Verifications in API Server for VN ctest-vn1-30521313 passed 2025-07-22 18:16:32,382 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:32,394 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/0271c8f9-85eb-423b-a83d-4fd53f1fd5c7 2025-07-22 18:16:32,404 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/0271c8f9-85eb-423b-a83d-4fd53f1fd5c7 2025-07-22 18:16:32,414 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/9c7748c6-3e4b-4d98-8d39-929b1e6ef93d 2025-07-22 18:16:32,429 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-42431603', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12128686049028165315', 'uuid-lslong': '10377618479864754660'}, 'enable': 'true', 'created': '2025-07-22T18:16:31', 'last-modified': '2025-07-22T18:16:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.769253'} 2025-07-22 18:16:32,437 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:16:32,444 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-42431603', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12128686049028165315', 'uuid-lslong': '10377618479864754660'}, 'enable': 'true', 'created': '2025-07-22T18:16:31', 'last-modified': '2025-07-22T18:16:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.783987'} 2025-07-22 18:16:32,454 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:16:32,463 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-42431603', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12128686049028165315', 'uuid-lslong': '10377618479864754660'}, 'enable': 'true', 'created': '2025-07-22T18:16:31', 'last-modified': '2025-07-22T18:16:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.802356'} 2025-07-22 18:16:32,471 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:16:32,471 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-30521313 passed 2025-07-22 18:16:32,472 - DEBUG - ====Verifying policy data for ctest-vn1-30521313 in API_Server ====== 2025-07-22 18:16:32,472 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 18:16:32,479 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 18:16:32,494 - DEBUG - Requesting: http://10.0.0.254:8082/project/f557f3c8-3611-41ae-9e19-02699aa8ef0a 2025-07-22 18:16:32,602 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a851bfb3-788f-4ec3-9004-b50642de31e4 2025-07-22 18:16:32,613 - DEBUG - =>VN ctest-vn1-30521313 has no policy to be verified 2025-07-22 18:16:32,613 - DEBUG - Verifying the vn in opserver 2025-07-22 18:16:32,613 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 virtual network link through opserver 10.0.0.254 2025-07-22 18:16:32,613 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 18:16:33,010 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 18:16:33,070 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313?flat'} 2025-07-22 18:16:33,070 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 virtual network link through opserver 10.0.0.50 2025-07-22 18:16:33,071 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 18:16:33,362 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 18:16:33,422 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313?flat'} 2025-07-22 18:16:33,422 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 virtual network link through opserver 10.0.0.23 2025-07-22 18:16:33,422 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 18:16:33,716 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 18:16:33,775 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313?flat'} 2025-07-22 18:16:33,776 - INFO - Validated that VN default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is found in opserver 2025-07-22 18:16:33,798 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 18:16:33,810 - DEBUG - VRF ids for VN ctest-vn1-30521313: {} 2025-07-22 18:16:33,924 - DEBUG - Services list from nova: [, , , ] 2025-07-22 18:16:35,092 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 18:16:36,615 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1) 2025-07-22 18:16:36,615 - INFO - Waiting for VM ctest-vn1_vm1-68660272 to be up.. 2025-07-22 18:16:36,720 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-07-22 18:16:41,815 - DEBUG - VM is in ACTIVE state now 2025-07-22 18:16:41,815 - INFO - VM name : ctest-vn1_vm1-68660272 2025-07-22 18:16:41,896 - DEBUG - VM ctest-vn1_vm1-68660272 ID is a6dce570-ae5b-45cd-a852-7d3f0a7009ab 2025-07-22 18:16:41,919 - DEBUG - VM ctest-vn1_vm1-68660272 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 18:16:41,999 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a6dce570-ae5b-45cd-a852-7d3f0a7009ab 2025-07-22 18:16:42,294 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a6dce570-ae5b-45cd-a852-7d3f0a7009ab 2025-07-22 18:16:42,328 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/e4eea2f0-f747-4875-9c7e-2d3bc61dda2e 2025-07-22 18:16:45,527 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1029ms') 2025-07-22 18:16:45,527 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:16:45,545 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:16:49,623 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-07-22 18:16:49,623 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:16:49,640 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:16:53,724 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 18:16:53,725 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:16:53,744 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:16:57,817 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms') 2025-07-22 18:16:57,817 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:16:57,836 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:17:01,919 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 18:17:01,920 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:17:01,939 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:17:06,007 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-07-22 18:17:06,007 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:17:06,026 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:17:10,107 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 18:17:10,107 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:17:10,125 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:17:14,205 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-07-22 18:17:14,205 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 failed! 2025-07-22 18:17:14,224 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-30521313 is 192.168.1.1 and allocation pool is NOT set 2025-07-22 18:17:16,288 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.79 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.73 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 2.731/4.759/6.788/2.028 ms') 2025-07-22 18:17:16,288 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-68660272 passed 2025-07-22 18:17:16,368 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:16,368 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-68660272, IP 192.168.1.3, Port 22 2025-07-22 18:17:16,425 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 18:17:16,517 - DEBUG - VM ctest-vn1_vm1-68660272 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 18:17:21,518 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:21,519 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-68660272, IP 192.168.1.3, Port 22 2025-07-22 18:17:21,592 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 18:17:21,696 - DEBUG - VM ctest-vn1_vm1-68660272 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 18:17:26,696 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:26,696 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-68660272, IP 192.168.1.3, Port 22 2025-07-22 18:17:26,763 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 18:17:26,855 - DEBUG - VM ctest-vn1_vm1-68660272 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 18:17:31,856 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:31,856 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-68660272, IP 192.168.1.3, Port 22 2025-07-22 18:17:31,923 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 18:17:32,018 - DEBUG - VM ctest-vn1_vm1-68660272 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 18:17:37,020 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:37,020 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-68660272, IP 192.168.1.3, Port 22 2025-07-22 18:17:37,187 - DEBUG - VM ctest-vn1_vm1-68660272 is ready for SSH connections 2025-07-22 18:17:37,187 - INFO - Waiting for VM ctest-vn1_vm2-70485352 to be up.. 2025-07-22 18:17:37,278 - DEBUG - VM is in ACTIVE state now 2025-07-22 18:17:37,278 - INFO - VM name : ctest-vn1_vm2-70485352 2025-07-22 18:17:37,371 - DEBUG - VM ctest-vn1_vm2-70485352 ID is bad90e88-738b-4063-8d24-0441167dc0e5 2025-07-22 18:17:37,371 - DEBUG - VM ctest-vn1_vm2-70485352 launched on Node an-jenkins-deploy-platform-ansible-os-3693-1 2025-07-22 18:17:37,472 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/bad90e88-738b-4063-8d24-0441167dc0e5 2025-07-22 18:17:37,484 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/fc396021-6b8c-42c7-abef-bcf2d63e5b7b 2025-07-22 18:17:38,686 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.36 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.75 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 3.358/3.551/3.745/0.193 ms') 2025-07-22 18:17:38,686 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm2-70485352 passed 2025-07-22 18:17:38,770 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:38,770 - DEBUG - Waiting to SSH to VM ctest-vn1_vm2-70485352, IP 192.168.1.4, Port 22 2025-07-22 18:17:38,938 - DEBUG - VM ctest-vn1_vm2-70485352 is ready for SSH connections 2025-07-22 18:17:38,938 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:38,938 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 192.168.1.4, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 18:17:38,938 - DEBUG - ping -s 56 -c 3 -W 1 192.168.1.4 2025-07-22 18:17:59,591 - DEBUG - PING 192.168.1.4 (192.168.1.4) 56(84) bytes of data. 64 bytes from 192.168.1.4: icmp_req=1 ttl=64 time=14.0 ms 64 bytes from 192.168.1.4: icmp_req=2 ttl=64 time=1.42 ms 64 bytes from 192.168.1.4: icmp_req=3 ttl=64 time=1.28 ms --- 192.168.1.4 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2005ms rtt min/avg/max/mdev = 1.284/5.584/14.048/5.985 ms 2025-07-22 18:17:59,591 - INFO - Ping to IP 192.168.1.4 from VM ctest-vn1_vm1-68660272 passed 2025-07-22 18:17:59,591 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 18:17:59,591 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 192.168.1.3, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 18:17:59,591 - DEBUG - ping -s 56 -c 3 -W 1 192.168.1.3 2025-07-22 18:18:41,995 - DEBUG - PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data. 64 bytes from 192.168.1.3: icmp_req=1 ttl=64 time=7.79 ms 64 bytes from 192.168.1.3: icmp_req=2 ttl=64 time=1.17 ms 64 bytes from 192.168.1.3: icmp_req=3 ttl=64 time=1.26 ms --- 192.168.1.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2005ms rtt min/avg/max/mdev = 1.171/3.409/7.793/3.100 ms 2025-07-22 18:18:41,995 - INFO - Ping to IP 192.168.1.3 from VM ctest-vn1_vm2-70485352 passed 2025-07-22 18:18:42,004 - INFO - Active control node from the Agent 10.0.0.27 is 10.0.0.23 2025-07-22 18:18:42,004 - INFO - Stoping the Control service in 10.0.0.23 2025-07-22 18:18:42,424 - INFO - 10.0.0.23 2025-07-22 18:18:42,424 - INFO - control:inactive 2025-07-22 18:18:42,424 - INFO - Contrail services control are down on nodes 10.0.0.23 2025-07-22 18:18:42,427 - INFO - Deleting VM ctest-vn1_vm2-70485352 2025-07-22 18:18:42,493 - INFO - Deleting VM ctest-vn1_vm1-68660272 2025-07-22 18:18:42,556 - INFO - Deleting VN ctest-vn1-30521313 2025-07-22 18:18:42,600 - DEBUG - VN a851bfb3-788f-4ec3-9004-b50642de31e4 still in use: Unable to complete operation on network a851bfb3-788f-4ec3-9004-b50642de31e4. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-ebc314e6-59cf-4a3b-a4f0-d6fef8dee834'] 2025-07-22 18:18:42,600 - WARNING - Deleting VN ctest-vn1-30521313 failed..Will retry 2025-07-22 18:18:44,776 - DEBUG - Response for deleting network () 2025-07-22 18:18:44,776 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/0271c8f9-85eb-423b-a83d-4fd53f1fd5c7 2025-07-22 18:18:44,782 - DEBUG - Response Code: 404 2025-07-22 18:18:44,782 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 18:18:44,788 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 18:18:44,860 - DEBUG - Requesting: http://10.0.0.254:8082/project/f557f3c8-3611-41ae-9e19-02699aa8ef0a 2025-07-22 18:18:44,961 - INFO - Validated that VN ctest-vn1-30521313 is not found in API Server 2025-07-22 18:18:44,974 - DEBUG - VN ctest-vn1-30521313 is not present in Agent 10.0.0.19 2025-07-22 18:18:44,990 - DEBUG - VN ctest-vn1-30521313 is not present in Agent 10.0.0.27 2025-07-22 18:18:44,990 - INFO - Validated that VN ctest-vn1-30521313 is not in any agent 2025-07-22 18:18:46,285 - ERROR - NameError Python 3.6.8: /usr/bin/python3 Tue Jul 22 18:18:42 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_control_node_switchover(self=) 652 'contrail-control', [active_controller_host_ip], 653 container='control') 654 sleep(5) 655 656 # Check the control node shifted to other control node sleep undefined NameError: name 'sleep' is not defined __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Name not found globally.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("name 'sleep' is not defined",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 654, in test_control_node_switchover sleep(5) NameError: name 'sleep' is not defined 2025-07-22 18:18:46,286 - DEBUG - Skipping xmpp flap check 2025-07-22 18:18:46,286 - INFO - 2025-07-22 18:18:46,286 - INFO - END TEST : test_control_node_switchover : FAILED[0:02:17] 2025-07-22 18:18:46,286 - INFO - -------------------------------------------------------------------------------- 2025-07-22 18:18:46,289 - INFO - ================================================================================ 2025-07-22 18:18:46,289 - INFO - STARTING TEST : test_process_restart_in_policy_between_vns 2025-07-22 18:18:46,289 - INFO - TEST DESCRIPTION : Test to validate that with policy having rule to check icmp fwding between VMs on different VNs , ping between VMs should pass with process restarts 1. Pick 2 VN's from resource pool which has one VM each 2. Create policy with icmp allow rule between those VN's and bind it networks 3. Ping from one VM to another VM 4. Restart process 'vrouter' and 'control' on setup 5. Ping again between VM's after process restart Pass criteria: Step 2,3,4 and 5 should pass 2025-07-22 18:18:47,554 - DEBUG - Skipping xmpp flap check 2025-07-22 18:18:47,554 - INFO - Initial checks done. Running the testcase now 2025-07-22 18:18:47,554 - INFO - 2025-07-22 18:18:47,564 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-39586739, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn2-03449554, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-22 18:18:47,880 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn2-03449554, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-39586739, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-22 18:18:48,456 - INFO - Created VN ctest-vn1-39586739, UUID :286afb55-dcd1-4b75-8e6e-ac451253b0b6 2025-07-22 18:18:48,814 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 18:18:48,821 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 18:18:48,836 - DEBUG - Requesting: http://10.0.0.254:8082/project/f557f3c8-3611-41ae-9e19-02699aa8ef0a 2025-07-22 18:18:48,951 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/286afb55-dcd1-4b75-8e6e-ac451253b0b6 2025-07-22 18:18:48,967 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/286afb55-dcd1-4b75-8e6e-ac451253b0b6 2025-07-22 18:18:48,978 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/608d8a12-3737-4b46-b86f-0863c6419495 2025-07-22 18:18:48,988 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/608d8a12-3737-4b46-b86f-0863c6419495 2025-07-22 18:18:48,997 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/5537bdf0-d66b-43ac-b164-76bd0f6c9f69 2025-07-22 18:18:49,006 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:18:49,006 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/286afb55-dcd1-4b75-8e6e-ac451253b0b6 2025-07-22 18:18:49,017 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/608d8a12-3737-4b46-b86f-0863c6419495 2025-07-22 18:18:49,035 - INFO - Verified VN network id 12 for VN 286afb55-dcd1-4b75-8e6e-ac451253b0b6 2025-07-22 18:18:49,035 - INFO - Verifications in API Server for VN ctest-vn1-39586739 passed 2025-07-22 18:18:49,036 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/286afb55-dcd1-4b75-8e6e-ac451253b0b6 2025-07-22 18:18:49,047 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/608d8a12-3737-4b46-b86f-0863c6419495 2025-07-22 18:18:49,057 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/608d8a12-3737-4b46-b86f-0863c6419495 2025-07-22 18:18:49,066 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/5537bdf0-d66b-43ac-b164-76bd0f6c9f69 2025-07-22 18:18:49,080 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-39586739', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-42431603', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2912416455243942773', 'uuid-lslong': '10263330013484134582'}, 'enable': 'true', 'created': '2025-07-22T18:18:48', 'last-modified': '2025-07-22T18:18:48', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.458193'} 2025-07-22 18:18:49,087 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:18:49,094 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-39586739', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-42431603', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2912416455243942773', 'uuid-lslong': '10263330013484134582'}, 'enable': 'true', 'created': '2025-07-22T18:18:48', 'last-modified': '2025-07-22T18:18:48', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.465847'} 2025-07-22 18:18:49,102 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 18:18:49,106 - INFO - Deleting VN ctest-vn1-39586739 2025-07-22 18:18:49,299 - INFO - Deleted policy policy2 2025-07-22 18:18:49,366 - INFO - Deleted policy policy1 2025-07-22 18:18:50,637 - ERROR - AttributeError Python 3.6.8: /usr/bin/python3 Tue Jul 22 18:18:49 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_process_restart_in_policy_between_vns(self=) 397 connections=self.connections)) 398 vn1_fixture = self.create_vn(vn1_name, vn1_subnets, option='contrail') 399 assert vn1_fixture.verify_on_setup() 400 vn1_fixture.bind_policies( 401 [policy1_fixture.policy_fq_name], vn1_fixture.vn_id) vn1_fixture = vn1_fixture.verify_on_setup = > /contrail-test/fixtures/vn_test.py in verify_on_setup(self=) 579 "One or more verifications in API Server for VN %s failed" % (self.vn_name)) 580 return result 581 if not self.verify_vn_in_control_nodes(): 582 result = result and False 583 self.logger.error( self = self.verify_vn_in_control_nodes = .deco_retry..f_retry of > /contrail-test/tcutils/util.py in f_retry(*args=(,), **kwargs={}) 82 mtries, mdelay = tries, delay # make mutable 83 84 result = f(*args, **kwargs) # first attempt 85 rv = result 86 final = False result undefined f = args = (,) kwargs = {} /contrail-test/fixtures/vn_test.py in verify_vn_in_control_nodes(self=) 927 for cn in self.inputs.bgp_ips: 928 cn_config_vn_obj = self.cn_inspect[cn].get_cn_config_vn( 929 vn_name=self.vn_name, project=self.project_name, domain=self.domain_name) 930 if not cn_config_vn_obj: 931 self.logger.warn('Control-node %s does not have VN %s info ' % vn_name undefined self = self.vn_name = 'ctest-vn1-39586739' project undefined self.project_name = 'ctest-TestBasicVMVN0-42431603' domain undefined self.domain_name = 'default-domain' /contrail-test/tcutils/control/cn_introspect_utils.py in get_cn_config_vn(self=, domain='default-domain', project='ctest-TestBasicVMVN0-42431603', vn_name='ctest-vn1-39586739') 139 def get_cn_config_vn(self, domain='default-domain', project='admin', vn_name='default-virtual-network'): 140 m = 'virtual-network:' + domain + ':' + project + ':' + vn_name 141 return self._get_if_map_table_entry(m) 142 143 def get_cn_config_fip_pool(self, domain='default-domain', project='admin', vn_name='default-virtual-network', fip_pool_name='default-floating-ip-pool'): self = self._get_if_map_table_entry = > m = 'virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-39586739' /contrail-test/tcutils/control/cn_introspect_utils.py in _get_if_map_table_entry(self=, match='virtual-network:default-domain:ctest-TestBasicVMVN0-42431603:ctest-vn1-39586739') 37 new_table_req = 'Snh_IFMapTableShowReq?table_name=' + table_name.group(1) + '&search_string=' + match 38 p = self.dict_get(new_table_req) 39 xp = p.xpath('./IFMapTableShowResp/ifmap_db/list/IFMapNodeShowInfo') 40 if not xp: 41 # sometime ./xpath dosen't work; work around xp undefined p = None p.xpath undefined AttributeError: 'NoneType' object has no attribute 'xpath' __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Attribute not found.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("'NoneType' object has no attribute 'xpath'",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 399, in test_process_restart_in_policy_between_vns assert vn1_fixture.verify_on_setup() File "/contrail-test/fixtures/vn_test.py", line 581, in verify_on_setup if not self.verify_vn_in_control_nodes(): File "/contrail-test/tcutils/util.py", line 84, in f_retry result = f(*args, **kwargs) # first attempt File "/contrail-test/fixtures/vn_test.py", line 929, in verify_vn_in_control_nodes vn_name=self.vn_name, project=self.project_name, domain=self.domain_name) File "/contrail-test/tcutils/control/cn_introspect_utils.py", line 141, in get_cn_config_vn return self._get_if_map_table_entry(m) File "/contrail-test/tcutils/control/cn_introspect_utils.py", line 39, in _get_if_map_table_entry xp = p.xpath('./IFMapTableShowResp/ifmap_db/list/IFMapNodeShowInfo') AttributeError: 'NoneType' object has no attribute 'xpath' 2025-07-22 18:18:50,638 - DEBUG - Skipping xmpp flap check 2025-07-22 18:18:50,638 - INFO - 2025-07-22 18:18:50,638 - INFO - END TEST : test_process_restart_in_policy_between_vns : FAILED[0:00:04] 2025-07-22 18:18:50,638 - INFO - -------------------------------------------------------------------------------- 2025-07-22 18:18:50,641 - INFO - ================================================================================ 2025-07-22 18:18:50,641 - INFO - STARTING TEST : test_underlay_broadcast_traffic_handling 2025-07-22 18:18:50,641 - INFO - TEST DESCRIPTION : Test the underlay brocast traffic handling by vrouter. (Bug-1545229). 1. Send broadcast traffic from one compute node. 2. Other compute in same subnet should receive that traffic. 3. Receiving compute should treat this traffic as underlay. 4. Compute should not replicate the packet and send the copy back. Pass criteria: Step 3-4 should pass Maintainer : chhandak@juniper.net 2025-07-22 18:18:51,893 - DEBUG - Skipping xmpp flap check 2025-07-22 18:18:51,894 - INFO - Initial checks done. Running the testcase now 2025-07-22 18:18:51,894 - INFO - 2025-07-22 18:18:52,279 - INFO - Agent interface name: ens6 2025-07-22 18:18:52,346 - DEBUG - Executing command: sudo tcpdump -nni ens6 -U host 10.20.0.255 -w /tmp/ens6_ctest-random-27505310.pcap 2025-07-22 18:18:53,957 - ERROR - NameError Python 3.6.8: /usr/bin/python3 Tue Jul 22 18:18:52 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_underlay_broadcast_traffic_handling(self=) 914 compute_password, comp_intf, filters, self.logger) 915 916 sleep(5) 917 918 # Ping broadcast address sleep undefined NameError: name 'sleep' is not defined __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Name not found globally.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("name 'sleep' is not defined",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 916, in test_underlay_broadcast_traffic_handling sleep(5) NameError: name 'sleep' is not defined 2025-07-22 18:18:53,957 - DEBUG - Skipping xmpp flap check 2025-07-22 18:18:53,957 - INFO - 2025-07-22 18:18:53,958 - INFO - END TEST : test_underlay_broadcast_traffic_handling : FAILED[0:00:03] 2025-07-22 18:18:53,958 - INFO - -------------------------------------------------------------------------------- 2025-07-22 18:18:54,722 - INFO - Deleted project: ctest-TestBasicVMVN0-42431603, ID : f557f3c8-3611-41ae-9e19-02699aa8ef0a