2025-08-06 08:32:59,632 - INFO - Domain Default found not creating 2025-08-06 08:32:59,853 - INFO - Project ctest-TestStaticRouteTables-49553622 not found, creating it 2025-08-06 08:33:00,422 - INFO - Created Project:ctest-TestStaticRouteTables-49553622, ID : 3732fbac-4ac8-402a-9373-cde9ee3ed5d0 2025-08-06 08:33:02,058 - INFO - ================================================================================ 2025-08-06 08:33:02,058 - INFO - STARTING TEST : test_interface_static_table 2025-08-06 08:33:02,059 - INFO - TEST DESCRIPTION : Description: Validate interface static table Test steps: 1. Creating vm's - vm1 and vm2 and middle vm in networks vn1 and vn2. 2. Creating interface static table betn left and right vm 3. Apply static table to the middle ports. 4. Checking for ping and tcp traffic between vm1 and vm2. Pass criteria: Ping between the VMs should be successful and route should be in the agent 2025-08-06 08:33:03,336 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 08:33:03,336 - INFO - Initial checks done. Running the testcase now 2025-08-06 08:33:03,336 - INFO - 2025-08-06 08:33:04,028 - DEBUG - Response for create_network : {'network': {'id': '0057ce4b-835b-4b2e-9a9e-c7ac86c163d0', 'name': 'ctest-vn-57585309', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-vn-57585309'], 'port_security_enabled': True, 'description': ''}} 2025-08-06 08:33:04,411 - DEBUG - Response for create_subnet : {'subnet': {'id': '028fbc2b-fd2f-40df-af97-4abf01ea5708', 'name': '', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'network_id': '0057ce4b-835b-4b2e-9a9e-c7ac86c163d0', 'ip_version': 4, 'cidr': '213.17.24.64/26', 'allocation_pools': [{'start': '213.17.24.66', 'end': '213.17.24.126'}], 'gateway_ip': '213.17.24.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '213.17.24.66', 'tags': [], 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0'}} 2025-08-06 08:33:04,445 - INFO - Created VN ctest-vn-57585309 2025-08-06 08:33:04,460 - DEBUG - VN ctest-vn-57585309 UUID is 0057ce4b-835b-4b2e-9a9e-c7ac86c163d0 2025-08-06 08:33:04,890 - DEBUG - Response for create_network : {'network': {'id': '63a133e2-4d63-478d-89e5-e93846464531', 'name': 'ctest-vn-84289567', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-vn-84289567'], 'port_security_enabled': True, 'description': ''}} 2025-08-06 08:33:05,146 - DEBUG - Response for create_subnet : {'subnet': {'id': '7196f8a9-4a6d-435d-acef-b7ec5b359f1c', 'name': '', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'network_id': '63a133e2-4d63-478d-89e5-e93846464531', 'ip_version': 4, 'cidr': '49.160.6.192/26', 'allocation_pools': [{'start': '49.160.6.194', 'end': '49.160.6.254'}], 'gateway_ip': '49.160.6.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '49.160.6.194', 'tags': [], 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0'}} 2025-08-06 08:33:05,172 - INFO - Created VN ctest-vn-84289567 2025-08-06 08:33:05,187 - DEBUG - VN ctest-vn-84289567 UUID is 63a133e2-4d63-478d-89e5-e93846464531 2025-08-06 08:33:05,410 - DEBUG - Services list from nova: [, , , ] 2025-08-06 08:33:06,595 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3761-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3761-1) 2025-08-06 08:33:08,268 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3761-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3761-2) 2025-08-06 08:33:08,269 - INFO - Waiting for VM ctest-TestStaticRouteTables-49553622-09664762 to be up.. 2025-08-06 08:33:08,385 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-06 08:33:13,483 - DEBUG - VM is in ACTIVE state now 2025-08-06 08:33:13,484 - INFO - VM name : ctest-TestStaticRouteTables-49553622-09664762 2025-08-06 08:33:13,593 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 ID is d37b7e82-b640-4698-9d2b-de428276cacf 2025-08-06 08:33:13,619 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 launched on Node an-jenkins-deploy-platform-ansible-os-3761-1 2025-08-06 08:33:13,732 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/d37b7e82-b640-4698-9d2b-de428276cacf 2025-08-06 08:33:14,033 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/d37b7e82-b640-4698-9d2b-de428276cacf 2025-08-06 08:33:14,089 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/805bc529-d5e1-46dc-a440-328bc8b51044 2025-08-06 08:33:17,339 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-08-06 08:33:17,339 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:17,358 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:21,438 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-08-06 08:33:21,438 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:21,459 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:25,535 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms') 2025-08-06 08:33:25,535 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:25,554 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:29,627 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-08-06 08:33:29,627 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:29,644 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:33,732 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1024ms') 2025-08-06 08:33:33,732 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:33,754 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:37,824 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms') 2025-08-06 08:33:37,824 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:37,845 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:41,913 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-08-06 08:33:41,914 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 failed! 2025-08-06 08:33:41,931 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:33:44,000 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=11.6 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=6.31 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 6.309/8.961/11.614/2.652 ms') 2025-08-06 08:33:44,000 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-09664762 passed 2025-08-06 08:33:44,121 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:33:44,121 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-09664762, IP 213.17.24.67, Port 22 2025-08-06 08:33:44,177 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:33:44,275 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:33:49,276 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:33:49,276 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-09664762, IP 213.17.24.67, Port 22 2025-08-06 08:33:49,342 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:33:49,445 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:33:54,446 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:33:54,446 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-09664762, IP 213.17.24.67, Port 22 2025-08-06 08:33:54,514 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:33:54,609 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:33:59,610 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:33:59,610 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-09664762, IP 213.17.24.67, Port 22 2025-08-06 08:33:59,669 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:33:59,760 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:34:04,761 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:34:04,762 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-09664762, IP 213.17.24.67, Port 22 2025-08-06 08:34:04,939 - DEBUG - VM ctest-TestStaticRouteTables-49553622-09664762 is ready for SSH connections 2025-08-06 08:34:04,939 - INFO - Waiting for VM ctest-TestStaticRouteTables-49553622-51473442 to be up.. 2025-08-06 08:34:05,045 - DEBUG - VM is in ACTIVE state now 2025-08-06 08:34:05,045 - INFO - VM name : ctest-TestStaticRouteTables-49553622-51473442 2025-08-06 08:34:05,141 - DEBUG - VM ctest-TestStaticRouteTables-49553622-51473442 ID is 2103c5c8-9f58-431d-bc53-16fe895a5bb3 2025-08-06 08:34:05,141 - DEBUG - VM ctest-TestStaticRouteTables-49553622-51473442 launched on Node an-jenkins-deploy-platform-ansible-os-3761-2 2025-08-06 08:34:05,252 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/2103c5c8-9f58-431d-bc53-16fe895a5bb3 2025-08-06 08:34:05,265 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/ae7069e8-c4bc-4719-8955-b3cebb20643a 2025-08-06 08:34:06,481 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.31 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.632 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.632/3.470/6.309/2.838 ms') 2025-08-06 08:34:06,481 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-51473442 passed 2025-08-06 08:34:06,598 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:34:06,598 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-51473442, IP 49.160.6.195, Port 22 2025-08-06 08:34:06,666 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:34:06,782 - DEBUG - VM ctest-TestStaticRouteTables-49553622-51473442 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:34:11,783 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:34:11,783 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-51473442, IP 49.160.6.195, Port 22 2025-08-06 08:34:11,954 - DEBUG - VM ctest-TestStaticRouteTables-49553622-51473442 is ready for SSH connections 2025-08-06 08:34:13,385 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3761-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3761-1) 2025-08-06 08:34:13,385 - INFO - Waiting for VM middle_vm1 to be up.. 2025-08-06 08:34:13,462 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-06 08:34:18,581 - DEBUG - VM is in ACTIVE state now 2025-08-06 08:34:18,581 - INFO - VM name : middle_vm1 2025-08-06 08:34:18,690 - DEBUG - VM middle_vm1 ID is d6ac2fda-b5bf-4824-a546-8af82231973a 2025-08-06 08:34:18,690 - DEBUG - VM middle_vm1 launched on Node an-jenkins-deploy-platform-ansible-os-3761-1 2025-08-06 08:34:18,896 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/d6ac2fda-b5bf-4824-a546-8af82231973a 2025-08-06 08:34:18,912 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/61557b24-15e1-47b2-886f-ba58e22c2da3 2025-08-06 08:34:18,924 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/bf588150-33b9-4668-9fa9-f7046daa4dc2 2025-08-06 08:34:22,203 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-08-06 08:34:22,203 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:22,220 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:26,296 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-08-06 08:34:26,296 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:26,316 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:30,390 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-08-06 08:34:30,390 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:30,407 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:34,492 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1024ms') 2025-08-06 08:34:34,492 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:34,509 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:38,594 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-08-06 08:34:38,594 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:38,614 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:42,689 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-08-06 08:34:42,690 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:42,707 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:46,785 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-08-06 08:34:46,785 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:46,809 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:50,878 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms') 2025-08-06 08:34:50,878 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:34:50,896 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-57585309 is 213.17.24.65 and allocation pool is NOT set 2025-08-06 08:34:52,964 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=4.80 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.646 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.646/2.722/4.799/2.076 ms') 2025-08-06 08:34:52,964 - INFO - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 passed 2025-08-06 08:34:53,081 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:34:53,082 - DEBUG - Waiting to SSH to VM middle_vm1, IP 213.17.24.68, Port 22 2025-08-06 08:34:53,150 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:34:53,266 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:34:58,267 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:34:58,267 - DEBUG - Waiting to SSH to VM middle_vm1, IP 213.17.24.68, Port 22 2025-08-06 08:34:58,325 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:34:58,418 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:35:03,420 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:35:03,420 - DEBUG - Waiting to SSH to VM middle_vm1, IP 213.17.24.68, Port 22 2025-08-06 08:35:03,477 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:35:03,568 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:35:08,569 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:35:08,569 - DEBUG - Waiting to SSH to VM middle_vm1, IP 213.17.24.68, Port 22 2025-08-06 08:35:08,624 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:35:08,720 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:35:13,721 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:35:13,721 - DEBUG - Waiting to SSH to VM middle_vm1, IP 213.17.24.68, Port 22 2025-08-06 08:35:13,896 - DEBUG - VM middle_vm1 is ready for SSH connections 2025-08-06 08:35:14,012 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:35:14,012 - DEBUG - Waiting to SSH to VM middle_vm1, IP 213.17.24.68, Port 22 2025-08-06 08:35:14,172 - DEBUG - VM middle_vm1 is ready for SSH connections 2025-08-06 08:35:14,173 - DEBUG - Running remote_cmd, Cmd : echo 1 > /proc/sys/net/ipv4/ip_forward, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-06 08:35:14,173 - DEBUG - echo 1 > /proc/sys/net/ipv4/ip_forward 2025-08-06 08:35:31,437 - DEBUG - None 2025-08-06 08:35:31,547 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-int_table_right-13087731'](UUID ebef3250-9457-42a2-8cf1-48cb909eacd4), prefixes : ['49.160.6.192/26'] 2025-08-06 08:35:31,626 - INFO - Added intf route table ebef3250-9457-42a2-8cf1-48cb909eacd4 to port bf588150-33b9-4668-9fa9-f7046daa4dc2 2025-08-06 08:35:31,767 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-int_table_left-44973997'](UUID fffab585-4413-4c6a-9bd3-7e7f38597c4e), prefixes : ['213.17.24.64/26'] 2025-08-06 08:35:31,842 - INFO - Added intf route table fffab585-4413-4c6a-9bd3-7e7f38597c4e to port 61557b24-15e1-47b2-886f-ba58e22c2da3 2025-08-06 08:35:34,849 - DEBUG - file copy failed retrying 2025-08-06 08:35:37,850 - DEBUG - file copy failed retrying 2025-08-06 08:36:18,918 - DEBUG - file transfer done ['/tmp/udpechoserver.py'] 2025-08-06 08:36:18,918 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:36:18,918 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoserver.py to 49.160.6.195 2025-08-06 08:36:18,918 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-45735152.pid --stats_file /tmp/server-ctest-random-45735152.stats 0<&- &> /tmp/server-ctest-random-45735152.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-08-06 08:36:18,919 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-45735152.pid --stats_file /tmp/server-ctest-random-45735152.stats 0<&- &> /tmp/server-ctest-random-45735152.log & 2025-08-06 08:36:19,405 - DEBUG - None 2025-08-06 08:36:55,411 - DEBUG - file transfer done ['/tmp/udpechoclient.py'] 2025-08-06 08:36:55,411 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:36:55,411 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoclient.py to 213.17.24.67 2025-08-06 08:36:55,411 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 49.160.6.195 --sport 8001 --dports 9001 --retry --pid_file /tmp/client-ctest-random-45735152.pid --stats_file /tmp/client-ctest-random-45735152.stats 0<&- &> /tmp/client-ctest-random-45735152.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-06 08:36:55,411 - DEBUG - nohup python /tmp/udpechoclient.py --servers 49.160.6.195 --sport 8001 --dports 9001 --retry --pid_file /tmp/client-ctest-random-45735152.pid --stats_file /tmp/client-ctest-random-45735152.stats 0<&- &> /tmp/client-ctest-random-45735152.log & 2025-08-06 08:36:55,850 - DEBUG - None 2025-08-06 08:36:55,850 - DEBUG - Sleeping for 7 seconds.. 2025-08-06 08:37:02,851 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-45735152.pid); sync; cat /tmp/client-ctest-random-45735152.stats, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-06 08:37:02,851 - DEBUG - kill $(cat /tmp/client-ctest-random-45735152.pid); sync; cat /tmp/client-ctest-random-45735152.stats 2025-08-06 08:37:03,455 - DEBUG - dport: 9001 - dst ip: 49.160.6.195 - sent: 2 - recv: 0 2025-08-06 08:37:03,455 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-45735152.pid); sync; cat /tmp/server-ctest-random-45735152.stats, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-08-06 08:37:03,455 - DEBUG - kill $(cat /tmp/server-ctest-random-45735152.pid); sync; cat /tmp/server-ctest-random-45735152.stats 2025-08-06 08:37:04,156 - DEBUG - cat: /tmp/server-ctest-random-45735152.pid: No such file or directory kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] cat: /tmp/server-ctest-random-45735152.stats: No such file or directory 2025-08-06 08:37:04,157 - INFO - Client - Sent: 2, Received: 0 2025-08-06 08:37:04,157 - INFO - Server - Sent: 0, Received: 0 2025-08-06 08:37:04,238 - INFO - Removed intf route table ebef3250-9457-42a2-8cf1-48cb909eacd4 from port bf588150-33b9-4668-9fa9-f7046daa4dc2 2025-08-06 08:37:04,314 - INFO - Removed intf route table fffab585-4413-4c6a-9bd3-7e7f38597c4e from port 61557b24-15e1-47b2-886f-ba58e22c2da3 2025-08-06 08:37:04,352 - INFO - Deleted Interface route table ebef3250-9457-42a2-8cf1-48cb909eacd4 2025-08-06 08:37:04,395 - INFO - Deleted Interface route table fffab585-4413-4c6a-9bd3-7e7f38597c4e 2025-08-06 08:37:04,395 - INFO - Deleting VM middle_vm1 2025-08-06 08:37:04,508 - INFO - Deleting VM ctest-TestStaticRouteTables-49553622-51473442 2025-08-06 08:37:04,622 - INFO - Deleting VM ctest-TestStaticRouteTables-49553622-09664762 2025-08-06 08:37:04,816 - INFO - Deleting VN ctest-vn-84289567 2025-08-06 08:37:04,888 - DEBUG - VN 63a133e2-4d63-478d-89e5-e93846464531 still in use: Unable to complete operation on network 63a133e2-4d63-478d-89e5-e93846464531. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-503a4ccf-ed64-446c-85ee-545521a76550'] 2025-08-06 08:37:04,888 - WARNING - Deleting VN ctest-vn-84289567 failed..Will retry 2025-08-06 08:37:07,073 - DEBUG - Response for deleting network () 2025-08-06 08:37:07,073 - INFO - Deleting VN ctest-vn-57585309 2025-08-06 08:37:07,308 - DEBUG - Response for deleting network () 2025-08-06 08:37:10,315 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Wed Aug 6 08:37:04 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/static_route_table/test_static_tables.py in test_interface_static_table(self=) 35 36 self.verify_traffic( 37 self.left_vm_fixture, self.right_vm_fixture, 'udp', sport=sport, dport=dport) 38 39 self.check_route_in_agent(expected_next_hops = 1) self = self.left_vm_fixture = self.right_vm_fixture = sport = 8001 dport = 9001 /contrail-test/common/base.py in verify_traffic(self=, src_vm_fixture=, dst_vm_fixture=, proto='udp', sport=8001, dport=9001, src_vn_fqname=None, dst_vn_fqname=None, af=None, fip_ip=None, expectation=True) 812 fip_ip=fip_ip) 813 self.sleep(7) 814 return self.stop_traffic(traffic_obj, expectation) 815 816 @classmethod self = self.stop_traffic = > traffic_obj = expectation = True /contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False) 796 assert False, "No drops observed in packets %s"%msg 797 elif expectation: 798 assert sent and recv, "Packets not %s"%msg 799 if recv*100/float(sent) < 90: 800 assert False, "Packets not %s"%msg sent = 2 recv = 0 msg = 'transferred between 213.17.24.67 and 49.160.6.195, proto udp sport 8001 and dport 9001' AssertionError: Packets not transferred between 213.17.24.67 and 49.160.6.195, proto udp sport 8001 and dport 9001 __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('Packets not transferred between 213.17.24.67 and 49.160.6.195, proto udp sport 8001 and dport 9001',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/static_route_table/test_static_tables.py", line 37, in test_interface_static_table self.left_vm_fixture, self.right_vm_fixture, 'udp', sport=sport, dport=dport) File "/contrail-test/common/base.py", line 814, in verify_traffic return self.stop_traffic(traffic_obj, expectation) File "/contrail-test/common/base.py", line 798, in stop_traffic assert sent and recv, "Packets not %s"%msg AssertionError: Packets not transferred between 213.17.24.67 and 49.160.6.195, proto udp sport 8001 and dport 9001 2025-08-06 08:37:10,333 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 08:37:10,333 - INFO - 2025-08-06 08:37:10,333 - INFO - END TEST : test_interface_static_table : FAILED[0:04:08] 2025-08-06 08:37:10,333 - INFO - -------------------------------------------------------------------------------- 2025-08-06 08:37:10,339 - INFO - ================================================================================ 2025-08-06 08:37:10,339 - INFO - STARTING TEST : test_network_route_table 2025-08-06 08:37:10,340 - INFO - TEST DESCRIPTION : Description: Validate network static table Test steps: 1. Creating vm's - vm1 and vm2 and middle vm in networks vn1 and vn2. 2. Creating network static table betn left and right vm 3. Apply static table to the middle ports. 4. Checking for ping and tcp traffic between vm1 and vm2. Pass criteria: Ping between the VMs should be successful and route should be in the agent 2025-08-06 08:37:11,609 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 08:37:11,609 - INFO - Initial checks done. Running the testcase now 2025-08-06 08:37:11,610 - INFO - 2025-08-06 08:37:11,905 - DEBUG - Response for create_network : {'network': {'id': 'ee7ee68d-a823-4a6b-9334-13186ef22927', 'name': 'ctest-vn-11825774', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-vn-11825774'], 'port_security_enabled': True, 'description': ''}} 2025-08-06 08:37:12,178 - DEBUG - Response for create_subnet : {'subnet': {'id': '617e489e-f115-4778-8597-4f56a84c1dd9', 'name': '', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'network_id': 'ee7ee68d-a823-4a6b-9334-13186ef22927', 'ip_version': 4, 'cidr': '176.211.163.64/26', 'allocation_pools': [{'start': '176.211.163.66', 'end': '176.211.163.126'}], 'gateway_ip': '176.211.163.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '176.211.163.66', 'tags': [], 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0'}} 2025-08-06 08:37:12,202 - INFO - Created VN ctest-vn-11825774 2025-08-06 08:37:12,215 - DEBUG - VN ctest-vn-11825774 UUID is ee7ee68d-a823-4a6b-9334-13186ef22927 2025-08-06 08:37:12,682 - DEBUG - Response for create_network : {'network': {'id': '5aeadaa8-d936-40d3-90d1-6fc723e251b3', 'name': 'ctest-vn-29752053', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-vn-29752053'], 'port_security_enabled': True, 'description': ''}} 2025-08-06 08:37:12,911 - DEBUG - Response for create_subnet : {'subnet': {'id': '7065707e-c7f9-466c-bf28-a379defb4a69', 'name': '', 'tenant_id': '3732fbac4ac8402a9373cde9ee3ed5d0', 'network_id': '5aeadaa8-d936-40d3-90d1-6fc723e251b3', 'ip_version': 4, 'cidr': '199.78.217.64/26', 'allocation_pools': [{'start': '199.78.217.66', 'end': '199.78.217.126'}], 'gateway_ip': '199.78.217.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '199.78.217.66', 'tags': [], 'project_id': '3732fbac4ac8402a9373cde9ee3ed5d0'}} 2025-08-06 08:37:12,950 - INFO - Created VN ctest-vn-29752053 2025-08-06 08:37:12,966 - DEBUG - VN ctest-vn-29752053 UUID is 5aeadaa8-d936-40d3-90d1-6fc723e251b3 2025-08-06 08:37:14,415 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3761-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3761-2) 2025-08-06 08:37:16,171 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3761-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3761-1) 2025-08-06 08:37:16,172 - INFO - Waiting for VM ctest-TestStaticRouteTables-49553622-37541348 to be up.. 2025-08-06 08:37:16,303 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-06 08:37:21,396 - DEBUG - VM is in ACTIVE state now 2025-08-06 08:37:21,397 - INFO - VM name : ctest-TestStaticRouteTables-49553622-37541348 2025-08-06 08:37:21,497 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 ID is 12c5884e-fbfa-4f5e-acb7-e9a963e093fb 2025-08-06 08:37:21,497 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 launched on Node an-jenkins-deploy-platform-ansible-os-3761-2 2025-08-06 08:37:21,591 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/12c5884e-fbfa-4f5e-acb7-e9a963e093fb 2025-08-06 08:37:21,604 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/f1497ce5-46ed-4aa7-887e-bfef76a6ac23 2025-08-06 08:37:24,862 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1033ms') 2025-08-06 08:37:24,862 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:24,880 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:28,962 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-08-06 08:37:28,963 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:28,982 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:33,055 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-08-06 08:37:33,055 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:33,073 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:37,158 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-08-06 08:37:37,158 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:37,180 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:41,251 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-08-06 08:37:41,251 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:41,269 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:45,344 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-08-06 08:37:45,345 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:45,361 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:49,444 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-08-06 08:37:49,444 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:49,463 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:53,535 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-08-06 08:37:53,535 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 failed! 2025-08-06 08:37:53,555 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:37:55,624 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=13.6 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.51 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 3.507/8.544/13.582/5.037 ms') 2025-08-06 08:37:55,624 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-37541348 passed 2025-08-06 08:37:55,745 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:37:55,745 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-37541348, IP 176.211.163.67, Port 22 2025-08-06 08:37:55,813 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:37:55,929 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:38:00,930 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:38:00,930 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-37541348, IP 176.211.163.67, Port 22 2025-08-06 08:38:01,001 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:38:01,091 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:38:06,093 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:38:06,093 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-37541348, IP 176.211.163.67, Port 22 2025-08-06 08:38:06,161 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:38:06,362 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:38:11,363 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:38:11,363 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-37541348, IP 176.211.163.67, Port 22 2025-08-06 08:38:11,419 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:38:11,538 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:38:16,540 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:38:16,540 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-37541348, IP 176.211.163.67, Port 22 2025-08-06 08:38:16,703 - DEBUG - VM ctest-TestStaticRouteTables-49553622-37541348 is ready for SSH connections 2025-08-06 08:38:16,703 - INFO - Waiting for VM ctest-TestStaticRouteTables-49553622-43515137 to be up.. 2025-08-06 08:38:16,816 - DEBUG - VM is in ACTIVE state now 2025-08-06 08:38:16,817 - INFO - VM name : ctest-TestStaticRouteTables-49553622-43515137 2025-08-06 08:38:16,923 - DEBUG - VM ctest-TestStaticRouteTables-49553622-43515137 ID is 822e5474-f1c1-4f0e-b34a-19bd203bea56 2025-08-06 08:38:16,923 - DEBUG - VM ctest-TestStaticRouteTables-49553622-43515137 launched on Node an-jenkins-deploy-platform-ansible-os-3761-1 2025-08-06 08:38:17,060 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/822e5474-f1c1-4f0e-b34a-19bd203bea56 2025-08-06 08:38:17,103 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/c5fc94ca-7443-4234-9276-f451c9195ec3 2025-08-06 08:38:18,353 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=2.51 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.60 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.511/2.554/2.597/0.043 ms') 2025-08-06 08:38:18,353 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestStaticRouteTables-49553622-43515137 passed 2025-08-06 08:38:18,471 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:38:18,471 - DEBUG - Waiting to SSH to VM ctest-TestStaticRouteTables-49553622-43515137, IP 199.78.217.67, Port 22 2025-08-06 08:38:18,639 - DEBUG - VM ctest-TestStaticRouteTables-49553622-43515137 is ready for SSH connections 2025-08-06 08:38:20,454 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3761-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3761-2) 2025-08-06 08:38:20,454 - INFO - Waiting for VM middle_vm1 to be up.. 2025-08-06 08:38:20,515 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-06 08:38:25,624 - DEBUG - VM is in ACTIVE state now 2025-08-06 08:38:25,624 - INFO - VM name : middle_vm1 2025-08-06 08:38:25,717 - DEBUG - VM middle_vm1 ID is 3fb69b3e-3ad7-4c60-b9c2-361654d3d92a 2025-08-06 08:38:25,717 - DEBUG - VM middle_vm1 launched on Node an-jenkins-deploy-platform-ansible-os-3761-2 2025-08-06 08:38:25,919 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine/3fb69b3e-3ad7-4c60-b9c2-361654d3d92a 2025-08-06 08:38:25,932 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/56e7b9f6-f4bf-4b31-b3e0-c781fd4c29e8 2025-08-06 08:38:25,945 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-machine-interface/f45c9ea7-4b6f-443b-a8f0-e9bc0e9b4002 2025-08-06 08:38:29,216 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-08-06 08:38:29,216 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:29,234 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:33,316 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-08-06 08:38:33,316 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:33,333 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:37,411 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-08-06 08:38:37,411 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:37,430 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:41,503 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-08-06 08:38:41,503 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:41,520 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:45,605 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms') 2025-08-06 08:38:45,605 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:45,625 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:49,699 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-08-06 08:38:49,699 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:49,723 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:53,795 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-08-06 08:38:53,795 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:53,811 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:38:57,891 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-08-06 08:38:57,892 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:38:57,909 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:39:01,981 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-08-06 08:39:01,981 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:39:01,999 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:39:06,079 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-08-06 08:39:06,079 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 failed! 2025-08-06 08:39:06,098 - DEBUG - Gateway for vn default-domain:ctest-TestStaticRouteTables-49553622:ctest-vn-11825774 is 176.211.163.65 and allocation pool is NOT set 2025-08-06 08:39:10,180 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=8.50 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1015ms\r\nrtt min/avg/max/mdev = 8.498/8.498/8.498/0.000 ms') 2025-08-06 08:39:10,180 - INFO - Ping to Metadata IP 169.254.0.4 of VM middle_vm1 passed 2025-08-06 08:39:10,299 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:10,299 - DEBUG - Waiting to SSH to VM middle_vm1, IP 176.211.163.68, Port 22 2025-08-06 08:39:10,365 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:39:10,474 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:39:15,475 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:15,475 - DEBUG - Waiting to SSH to VM middle_vm1, IP 176.211.163.68, Port 22 2025-08-06 08:39:15,541 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:39:15,642 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:39:20,643 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:20,643 - DEBUG - Waiting to SSH to VM middle_vm1, IP 176.211.163.68, Port 22 2025-08-06 08:39:20,709 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:39:20,811 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:39:25,811 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:25,812 - DEBUG - Waiting to SSH to VM middle_vm1, IP 176.211.163.68, Port 22 2025-08-06 08:39:25,882 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-06 08:39:26,002 - DEBUG - VM middle_vm1 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-06 08:39:31,004 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:31,004 - DEBUG - Waiting to SSH to VM middle_vm1, IP 176.211.163.68, Port 22 2025-08-06 08:39:31,181 - DEBUG - VM middle_vm1 is ready for SSH connections 2025-08-06 08:39:31,305 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:31,305 - DEBUG - Waiting to SSH to VM middle_vm1, IP 176.211.163.68, Port 22 2025-08-06 08:39:31,473 - DEBUG - VM middle_vm1 is ready for SSH connections 2025-08-06 08:39:31,473 - DEBUG - Running remote_cmd, Cmd : echo 1 > /proc/sys/net/ipv4/ip_forward, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-08-06 08:39:31,474 - DEBUG - echo 1 > /proc/sys/net/ipv4/ip_forward 2025-08-06 08:39:53,740 - DEBUG - None 2025-08-06 08:39:53,740 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:53,834 - INFO - Created NetworkRouteTable ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-network_table_left_to_right-09417288'](UUID 5b5c7dfc-dbc5-46ff-a0da-9d8e8e5832b0), prefixes : ['199.78.217.64/26'] 2025-08-06 08:39:53,946 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:39:54,074 - INFO - Created NetworkRouteTable ['default-domain', 'ctest-TestStaticRouteTables-49553622', 'ctest-network_table_right_to_left-80847840'](UUID 067281c3-6c74-4967-95f6-ffca932c7a6a), prefixes : ['176.211.163.64/26'] 2025-08-06 08:39:57,198 - DEBUG - file copy failed retrying 2025-08-06 08:40:00,199 - DEBUG - file copy failed retrying 2025-08-06 08:40:38,467 - DEBUG - file transfer done ['/tmp/udpechoserver.py'] 2025-08-06 08:40:38,467 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:40:38,467 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoserver.py to 199.78.217.67 2025-08-06 08:40:38,467 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-48625700.pid --stats_file /tmp/server-ctest-random-48625700.stats 0<&- &> /tmp/server-ctest-random-48625700.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-06 08:40:38,467 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-48625700.pid --stats_file /tmp/server-ctest-random-48625700.stats 0<&- &> /tmp/server-ctest-random-48625700.log & 2025-08-06 08:40:38,976 - DEBUG - None 2025-08-06 08:41:20,837 - DEBUG - file transfer done ['/tmp/udpechoclient.py'] 2025-08-06 08:41:20,837 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-06 08:41:20,837 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoclient.py to 176.211.163.67 2025-08-06 08:41:20,837 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 199.78.217.67 --sport 8001 --dports 9001 --retry --pid_file /tmp/client-ctest-random-48625700.pid --stats_file /tmp/client-ctest-random-48625700.stats 0<&- &> /tmp/client-ctest-random-48625700.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-08-06 08:41:20,837 - DEBUG - nohup python /tmp/udpechoclient.py --servers 199.78.217.67 --sport 8001 --dports 9001 --retry --pid_file /tmp/client-ctest-random-48625700.pid --stats_file /tmp/client-ctest-random-48625700.stats 0<&- &> /tmp/client-ctest-random-48625700.log & 2025-08-06 08:41:21,378 - DEBUG - None 2025-08-06 08:41:21,379 - DEBUG - Sleeping for 7 seconds.. 2025-08-06 08:41:28,379 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-48625700.pid); sync; cat /tmp/client-ctest-random-48625700.stats, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-08-06 08:41:28,379 - DEBUG - kill $(cat /tmp/client-ctest-random-48625700.pid); sync; cat /tmp/client-ctest-random-48625700.stats 2025-08-06 08:41:29,259 - DEBUG - dport: 9001 - dst ip: 199.78.217.67 - sent: 2 - recv: 0 2025-08-06 08:41:29,259 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-48625700.pid); sync; cat /tmp/server-ctest-random-48625700.stats, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-06 08:41:29,259 - DEBUG - kill $(cat /tmp/server-ctest-random-48625700.pid); sync; cat /tmp/server-ctest-random-48625700.stats 2025-08-06 08:41:29,912 - DEBUG - cat: /tmp/server-ctest-random-48625700.pid: No such file or directory kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] cat: /tmp/server-ctest-random-48625700.stats: No such file or directory 2025-08-06 08:41:29,912 - INFO - Client - Sent: 2, Received: 0 2025-08-06 08:41:29,912 - INFO - Server - Sent: 0, Received: 0 2025-08-06 08:41:30,030 - INFO - Removed network route table 5b5c7dfc-dbc5-46ff-a0da-9d8e8e5832b0 from network ee7ee68d-a823-4a6b-9334-13186ef22927 2025-08-06 08:41:30,174 - INFO - Removed network route table 067281c3-6c74-4967-95f6-ffca932c7a6a from network 5aeadaa8-d936-40d3-90d1-6fc723e251b3 2025-08-06 08:41:30,217 - INFO - Deleted Network route table 5b5c7dfc-dbc5-46ff-a0da-9d8e8e5832b0 2025-08-06 08:41:30,255 - INFO - Deleted Network route table 067281c3-6c74-4967-95f6-ffca932c7a6a 2025-08-06 08:41:30,255 - INFO - Deleting VM middle_vm1 2025-08-06 08:41:30,334 - INFO - Deleting VM ctest-TestStaticRouteTables-49553622-43515137 2025-08-06 08:41:30,403 - INFO - Deleting VM ctest-TestStaticRouteTables-49553622-37541348 2025-08-06 08:41:30,480 - INFO - Deleting VN ctest-vn-29752053 2025-08-06 08:41:30,538 - DEBUG - VN 5aeadaa8-d936-40d3-90d1-6fc723e251b3 still in use: Unable to complete operation on network 5aeadaa8-d936-40d3-90d1-6fc723e251b3. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-f7702403-57bd-4500-885e-345203d8cc51'] 2025-08-06 08:41:30,538 - WARNING - Deleting VN ctest-vn-29752053 failed..Will retry 2025-08-06 08:41:32,745 - DEBUG - Response for deleting network () 2025-08-06 08:41:32,746 - INFO - Deleting VN ctest-vn-11825774 2025-08-06 08:41:32,958 - DEBUG - Response for deleting network () 2025-08-06 08:41:35,690 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Wed Aug 6 08:41:29 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/static_route_table/test_static_tables.py in test_network_route_table(self=) 172 self.addCleanup(self.del_nw_route_table) 173 self.verify_traffic( 174 self.left_vm_fixture, self.right_vm_fixture, 'udp', sport=sport, dport=dport) 175 self.check_route_in_agent(expected_next_hops = 1) 176 self.unbind_network_table(self.vn1_fixture, self.vn2_fixture) self = self.left_vm_fixture = self.right_vm_fixture = sport = 8001 dport = 9001 /contrail-test/common/base.py in verify_traffic(self=, src_vm_fixture=, dst_vm_fixture=, proto='udp', sport=8001, dport=9001, src_vn_fqname=None, dst_vn_fqname=None, af=None, fip_ip=None, expectation=True) 812 fip_ip=fip_ip) 813 self.sleep(7) 814 return self.stop_traffic(traffic_obj, expectation) 815 816 @classmethod self = self.stop_traffic = > traffic_obj = expectation = True /contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False) 796 assert False, "No drops observed in packets %s"%msg 797 elif expectation: 798 assert sent and recv, "Packets not %s"%msg 799 if recv*100/float(sent) < 90: 800 assert False, "Packets not %s"%msg sent = 2 recv = 0 msg = 'transferred between 176.211.163.67 and 199.78.217.67, proto udp sport 8001 and dport 9001' AssertionError: Packets not transferred between 176.211.163.67 and 199.78.217.67, proto udp sport 8001 and dport 9001 __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('Packets not transferred between 176.211.163.67 a...99.78.217.67, proto udp sport 8001 and dport 9001',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/static_route_table/test_static_tables.py", line 174, in test_network_route_table self.left_vm_fixture, self.right_vm_fixture, 'udp', sport=sport, dport=dport) File "/contrail-test/common/base.py", line 814, in verify_traffic return self.stop_traffic(traffic_obj, expectation) File "/contrail-test/common/base.py", line 798, in stop_traffic assert sent and recv, "Packets not %s"%msg AssertionError: Packets not transferred between 176.211.163.67 and 199.78.217.67, proto udp sport 8001 and dport 9001 2025-08-06 08:41:35,707 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 08:41:35,707 - INFO - 2025-08-06 08:41:35,707 - INFO - END TEST : test_network_route_table : FAILED[0:04:25] 2025-08-06 08:41:35,708 - INFO - -------------------------------------------------------------------------------- 2025-08-06 08:41:36,795 - INFO - Deleted project: ctest-TestStaticRouteTables-49553622, ID : 3732fbac-4ac8-402a-9373-cde9ee3ed5d0