2025-07-22 17:17:07,375 - INFO - Domain Default found not creating 2025-07-22 17:17:07,518 - INFO - Project ctest-TestMtuProbe-84168175 not found, creating it 2025-07-22 17:17:08,096 - INFO - Created Project:ctest-TestMtuProbe-84168175, ID : bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:17:09,547 - INFO - ================================================================================ 2025-07-22 17:17:09,547 - INFO - STARTING TEST : test_mtu_probe_inter_node 2025-07-22 17:17:09,547 - INFO - TEST DESCRIPTION : Description: check traffic configuring different tcp_mtu_probe vlaues on vm MTU probing values: 0, 1, 2 0 - Disabled (default) 1 - Disabled by default, Enabled only when ICMP black hole detected 2 - Always enabled, using the initial MSS value of tcp_base_mss system internally enables bit values for point 1&2 For test purpose, Idea is to check traffic works fine on configuring values Vm's across compute node steps: 1. create 1 VN and launch 2 VMs across compute node 2. client and server VMs across compute node 3. configure tcp_mtu_probe to 0, 1, 2 4. send icmp traffic assert if ping fails 5. send tcp traffic assert if tcp traffic fails Pass criteria: 1. If icmp/tcp traffic work fine configuring tcp_mtu_probe values 2025-07-22 17:17:10,833 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-07-22 17:17:10,833 - INFO - Initial checks done. Running the testcase now 2025-07-22 17:17:10,833 - INFO - 2025-07-22 17:17:10,856 - DEBUG - Services list from nova: [, , , ] 2025-07-22 17:17:10,856 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:17:11,546 - DEBUG - Response for create_network : {'network': {'id': '7e487ebe-0829-4370-8dc4-8f8e293050a6', 'name': 'ctest-vn-90980520', 'tenant_id': 'bcea95588341461288c40d8752eeb9f2', 'project_id': 'bcea95588341461288c40d8752eeb9f2', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMtuProbe-84168175', 'ctest-vn-90980520'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:17:11,838 - DEBUG - Response for create_subnet : {'subnet': {'id': 'de34bc88-113f-4ce9-b5ad-e3d8aea38c7d', 'name': '', 'tenant_id': 'bcea95588341461288c40d8752eeb9f2', 'network_id': '7e487ebe-0829-4370-8dc4-8f8e293050a6', 'ip_version': 4, 'cidr': '104.12.205.192/26', 'allocation_pools': [{'start': '104.12.205.194', 'end': '104.12.205.254'}], 'gateway_ip': '104.12.205.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '104.12.205.194', 'tags': [], 'project_id': 'bcea95588341461288c40d8752eeb9f2'}} 2025-07-22 17:17:11,861 - INFO - Created VN ctest-vn-90980520 2025-07-22 17:17:11,881 - DEBUG - VN ctest-vn-90980520 UUID is 7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:11,953 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:17:12,244 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:17:12,271 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:17:12,288 - DEBUG - Requesting: http://10.0.0.254:8082/project/bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:17:12,387 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:12,399 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:12,410 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/408d62b0-b9c4-49c0-a68a-92d75d6cb45b 2025-07-22 17:17:12,419 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/408d62b0-b9c4-49c0-a68a-92d75d6cb45b 2025-07-22 17:17:12,426 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/e251d78b-b461-42a1-b97f-e0bda9ebb022 2025-07-22 17:17:12,434 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:17:12,434 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:12,444 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/408d62b0-b9c4-49c0-a68a-92d75d6cb45b 2025-07-22 17:17:12,460 - INFO - Verified VN network id 12 for VN 7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:12,460 - INFO - Verifications in API Server for VN ctest-vn-90980520 passed 2025-07-22 17:17:12,460 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:12,469 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/408d62b0-b9c4-49c0-a68a-92d75d6cb45b 2025-07-22 17:17:12,477 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/408d62b0-b9c4-49c0-a68a-92d75d6cb45b 2025-07-22 17:17:12,485 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/e251d78b-b461-42a1-b97f-e0bda9ebb022 2025-07-22 17:17:12,499 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestMtuProbe-84168175', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9099662401747895152', 'uuid-lslong': '10215447695522287782'}, 'enable': 'true', 'created': '2025-07-22T17:17:11', 'last-modified': '2025-07-22T17:17:11', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.725995'} 2025-07-22 17:17:12,506 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:17:12,512 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestMtuProbe-84168175', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9099662401747895152', 'uuid-lslong': '10215447695522287782'}, 'enable': 'true', 'created': '2025-07-22T17:17:11', 'last-modified': '2025-07-22T17:17:11', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.737459'} 2025-07-22 17:17:12,519 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:17:12,526 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestMtuProbe-84168175', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9099662401747895152', 'uuid-lslong': '10215447695522287782'}, 'enable': 'true', 'created': '2025-07-22T17:17:11', 'last-modified': '2025-07-22T17:17:11', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.751385'} 2025-07-22 17:17:12,533 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:17:12,533 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-90980520 passed 2025-07-22 17:17:12,533 - DEBUG - ====Verifying policy data for ctest-vn-90980520 in API_Server ====== 2025-07-22 17:17:12,534 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:17:12,540 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:17:12,553 - DEBUG - Requesting: http://10.0.0.254:8082/project/bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:17:12,650 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7e487ebe-0829-4370-8dc4-8f8e293050a6 2025-07-22 17:17:12,661 - DEBUG - =>VN ctest-vn-90980520 has no policy to be verified 2025-07-22 17:17:12,661 - DEBUG - Verifying the vn in opserver 2025-07-22 17:17:12,661 - DEBUG - Verifying the default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 virtual network link through opserver 10.0.0.254 2025-07-22 17:17:12,662 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:17:12,955 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:17:13,016 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520?flat'} 2025-07-22 17:17:13,016 - DEBUG - Verifying the default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 virtual network link through opserver 10.0.0.50 2025-07-22 17:17:13,016 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:17:13,307 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:17:13,363 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520?flat'} 2025-07-22 17:17:13,363 - DEBUG - Verifying the default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 virtual network link through opserver 10.0.0.23 2025-07-22 17:17:13,363 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:17:13,656 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:17:13,712 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520?flat'} 2025-07-22 17:17:13,712 - INFO - Validated that VN default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is found in opserver 2025-07-22 17:17:13,736 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 17:17:13,748 - DEBUG - VRF ids for VN ctest-vn-90980520: {} 2025-07-22 17:17:13,749 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:17:13,749 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:17:14,896 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 17:17:14,896 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:17:16,286 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1) 2025-07-22 17:17:16,286 - INFO - Waiting for VM ctest-TestMtuProbe-84168175-00542809 to be up.. 2025-07-22 17:17:16,383 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-07-22 17:17:21,464 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:17:21,464 - INFO - VM name : ctest-TestMtuProbe-84168175-00542809 2025-07-22 17:17:21,546 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 ID is 9667ff4d-5b90-43d0-87ad-b1e126b92584 2025-07-22 17:17:21,571 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 17:17:21,651 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/9667ff4d-5b90-43d0-87ad-b1e126b92584 2025-07-22 17:17:21,953 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/9667ff4d-5b90-43d0-87ad-b1e126b92584 2025-07-22 17:17:21,989 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/a2be4222-5352-4def-aa61-eec57e3ca534 2025-07-22 17:17:25,214 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-07-22 17:17:25,214 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:25,234 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:29,310 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-07-22 17:17:29,310 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:29,328 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:33,407 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-07-22 17:17:33,407 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:33,427 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:37,499 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1001ms') 2025-07-22 17:17:37,499 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:37,517 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:41,597 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-07-22 17:17:41,597 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:41,614 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:45,689 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-07-22 17:17:45,689 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:45,706 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:49,789 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-07-22 17:17:49,789 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:49,806 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:53,884 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-07-22 17:17:53,884 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:53,901 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:17:57,984 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 17:17:57,984 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 failed! 2025-07-22 17:17:58,002 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-90980520 is 104.12.205.193 and allocation pool is NOT set 2025-07-22 17:18:00,064 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.51 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.594 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.594/3.051/5.508/2.457 ms') 2025-07-22 17:18:00,065 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-00542809 passed 2025-07-22 17:18:00,184 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:18:00,184 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-00542809, IP 104.12.205.195, Port 22 2025-07-22 17:18:00,242 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:18:00,336 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:18:05,337 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:18:05,337 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-00542809, IP 104.12.205.195, Port 22 2025-07-22 17:18:05,397 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:18:05,501 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:18:10,502 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:18:10,502 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-00542809, IP 104.12.205.195, Port 22 2025-07-22 17:18:10,560 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:18:10,655 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:18:15,656 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:18:15,656 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-00542809, IP 104.12.205.195, Port 22 2025-07-22 17:18:15,713 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:18:15,817 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:18:20,818 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:18:20,818 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-00542809, IP 104.12.205.195, Port 22 2025-07-22 17:18:20,986 - DEBUG - VM ctest-TestMtuProbe-84168175-00542809 is ready for SSH connections 2025-07-22 17:18:20,987 - INFO - Waiting for VM ctest-TestMtuProbe-84168175-56568640 to be up.. 2025-07-22 17:18:21,071 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:18:21,071 - INFO - VM name : ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:18:21,159 - DEBUG - VM ctest-TestMtuProbe-84168175-56568640 ID is b18eea9c-c20d-453a-a66c-dc9c129587f5 2025-07-22 17:18:21,159 - DEBUG - VM ctest-TestMtuProbe-84168175-56568640 launched on Node an-jenkins-deploy-platform-ansible-os-3693-1 2025-07-22 17:18:21,244 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/b18eea9c-c20d-453a-a66c-dc9c129587f5 2025-07-22 17:18:21,257 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/42270404-7682-4347-ab52-d421d1e93cbd 2025-07-22 17:18:22,486 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=7.57 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.63 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 2.625/5.097/7.569/2.472 ms') 2025-07-22 17:18:22,486 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-56568640 passed 2025-07-22 17:18:22,605 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:18:22,605 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-56568640, IP 104.12.205.196, Port 22 2025-07-22 17:18:22,771 - DEBUG - VM ctest-TestMtuProbe-84168175-56568640 is ready for SSH connections 2025-07-22 17:18:22,771 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:18:22,771 - DEBUG - echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:18:38,869 - DEBUG - None 2025-07-22 17:18:38,869 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:18:38,869 - DEBUG - echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:19:17,920 - DEBUG - None 2025-07-22 17:19:17,920 - INFO - {'echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:19:17,920 - INFO - {'echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:19:17,921 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:19:17,921 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 104.12.205.196, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:19:17,921 - DEBUG - ping -s 56 -c 3 -W 1 104.12.205.196 2025-07-22 17:19:20,395 - DEBUG - PING 104.12.205.196 (104.12.205.196) 56(84) bytes of data. 64 bytes from 104.12.205.196: icmp_req=1 ttl=64 time=11.9 ms 64 bytes from 104.12.205.196: icmp_req=2 ttl=64 time=1.28 ms 64 bytes from 104.12.205.196: icmp_req=3 ttl=64 time=1.07 ms --- 104.12.205.196 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.072/4.761/11.925/5.066 ms 2025-07-22 17:19:20,396 - INFO - Ping to IP 104.12.205.196 from VM ctest-TestMtuProbe-84168175-00542809 passed 2025-07-22 17:19:20,396 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:19:20,396 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:20,396 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile & 2025-07-22 17:19:21,030 - DEBUG - total 132 drwxr-xr-x 11 ubuntu ubuntu 4096 Dec 29 2017 . drwxr-xr-x 3 root root 4096 Jan 4 2013 .. -rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history -rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout -rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache -rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins -rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla -rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh -rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh -rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0 -rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz -rw------- 1 root root 4695 Dec 17 2017 .viminfo -rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority -rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors 2025-07-22 17:19:21,030 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -ll 5201 > testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:21,030 - DEBUG - nohup nc -4 -q 2 -w 5 -ll 5201 > testfile & 2025-07-22 17:19:21,575 - DEBUG - None 2025-07-22 17:19:21,576 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:19:21,576 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:19:27,104 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.00088192 s, 113 kB/s 2025-07-22 17:19:27,104 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:27,104 - DEBUG - ls -l testfile 2025-07-22 17:19:27,434 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:27,434 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:30,434 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:30,435 - DEBUG - ls -l testfile 2025-07-22 17:19:30,756 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:30,757 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:33,757 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:33,758 - DEBUG - ls -l testfile 2025-07-22 17:19:34,102 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:34,102 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:37,103 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:37,103 - DEBUG - ls -l testfile 2025-07-22 17:19:37,435 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:37,436 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:40,436 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:40,436 - DEBUG - ls -l testfile 2025-07-22 17:19:40,799 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:40,799 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:43,799 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:43,800 - DEBUG - ls -l testfile 2025-07-22 17:19:44,142 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:44,142 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:47,143 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:47,143 - DEBUG - ls -l testfile 2025-07-22 17:19:47,474 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:47,474 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:50,475 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:50,475 - DEBUG - ls -l testfile 2025-07-22 17:19:50,825 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:50,825 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:53,826 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:53,827 - DEBUG - ls -l testfile 2025-07-22 17:19:54,192 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:54,192 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:19:57,194 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:19:57,194 - DEBUG - ls -l testfile 2025-07-22 17:19:57,543 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:19:57,543 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:00,544 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:00,544 - DEBUG - ls -l testfile 2025-07-22 17:20:00,878 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:00,878 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:00,878 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:20:00,878 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:20:00,878 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:20:03,552 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.00141731 s, 70.6 kB/s 2025-07-22 17:20:03,552 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:03,552 - DEBUG - ls -l testfile 2025-07-22 17:20:03,891 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:03,891 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:06,892 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:06,892 - DEBUG - ls -l testfile 2025-07-22 17:20:07,235 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:07,235 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:10,237 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:10,237 - DEBUG - ls -l testfile 2025-07-22 17:20:10,591 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:10,592 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:13,593 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:13,593 - DEBUG - ls -l testfile 2025-07-22 17:20:13,962 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:13,962 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:16,963 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:16,963 - DEBUG - ls -l testfile 2025-07-22 17:20:17,302 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:17,303 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:20,303 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:20,304 - DEBUG - ls -l testfile 2025-07-22 17:20:20,643 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:20,643 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:23,644 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:23,644 - DEBUG - ls -l testfile 2025-07-22 17:20:23,966 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:23,966 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:26,967 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:26,967 - DEBUG - ls -l testfile 2025-07-22 17:20:27,298 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:27,298 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:30,299 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:30,300 - DEBUG - ls -l testfile 2025-07-22 17:20:31,026 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:31,026 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:34,027 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:34,027 - DEBUG - ls -l testfile 2025-07-22 17:20:34,511 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:34,511 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:37,512 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:20:37,512 - DEBUG - ls -l testfile 2025-07-22 17:20:37,843 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:19 testfile 2025-07-22 17:20:37,843 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:20:37,843 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:20:37,844 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:20:37,844 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:20:38,401 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000980331 s, 102 kB/s nc: bind failed: Address already in use 2025-07-22 17:20:38,401 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:20:38,401 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:20:38,924 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:20:44,925 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:20:44,925 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:20:45,412 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:20:51,413 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:20:51,413 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:20:51,884 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:20:57,884 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:20:57,884 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:20:58,341 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:04,342 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:04,342 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:04,823 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:10,823 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:10,823 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:11,319 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:17,320 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:17,320 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:17,783 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:23,784 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:23,784 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:24,255 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:30,255 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:30,255 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:30,725 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:36,726 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:36,727 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:37,215 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:43,216 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:43,216 - DEBUG - nc -4 -q 2 -w 5 -p 10001 104.12.205.196 5201 < testfile 2025-07-22 17:21:43,705 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:21:43,706 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:21:43,706 - DEBUG - ls -l testfile 2025-07-22 17:21:44,046 - DEBUG - -rw-r--r-- 1 root root 100 Jul 22 17:20 testfile 2025-07-22 17:21:44,046 - INFO - File transfer verification for file size 100 passed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:21:44,046 - DEBUG - Running remote_cmd, Cmd : echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:44,046 - DEBUG - echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:21:44,490 - DEBUG - None 2025-07-22 17:21:44,490 - DEBUG - Running remote_cmd, Cmd : echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:21:44,490 - DEBUG - echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:21:44,968 - DEBUG - None 2025-07-22 17:21:44,968 - INFO - {'echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:21:44,968 - INFO - {'echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:21:44,968 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:21:44,968 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 104.12.205.196, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:44,968 - DEBUG - ping -s 56 -c 3 -W 1 104.12.205.196 2025-07-22 17:21:47,885 - DEBUG - PING 104.12.205.196 (104.12.205.196) 56(84) bytes of data. 64 bytes from 104.12.205.196: icmp_req=1 ttl=64 time=14.1 ms 64 bytes from 104.12.205.196: icmp_req=2 ttl=64 time=1.19 ms 64 bytes from 104.12.205.196: icmp_req=3 ttl=64 time=1.29 ms --- 104.12.205.196 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.193/5.552/14.172/6.095 ms 2025-07-22 17:21:47,885 - INFO - Ping to IP 104.12.205.196 from VM ctest-TestMtuProbe-84168175-00542809 passed 2025-07-22 17:21:47,885 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:21:47,885 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:21:47,885 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile & 2025-07-22 17:21:48,485 - DEBUG - total 132 drwxr-xr-x 11 ubuntu ubuntu 4096 Jul 22 17:21 . drwxr-xr-x 3 root root 4096 Jan 4 2013 .. -rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history -rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout -rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache -rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins -rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla -rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh -rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh -rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0 -rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz -rw------- 1 root root 4695 Dec 17 2017 .viminfo -rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority -rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors 2025-07-22 17:21:48,485 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -ll 5201 > testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:21:48,486 - DEBUG - nohup nc -4 -q 2 -w 5 -ll 5201 > testfile & 2025-07-22 17:21:49,037 - DEBUG - None 2025-07-22 17:21:49,037 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10002 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:21:49,037 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10002 104.12.205.196 5201 < testfile 2025-07-22 17:21:54,675 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.0112913 s, 8.9 kB/s 2025-07-22 17:21:54,676 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:21:54,676 - DEBUG - ls -l testfile 2025-07-22 17:21:55,015 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:21:55,015 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:21:58,016 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:21:58,016 - DEBUG - ls -l testfile 2025-07-22 17:21:58,382 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:21:58,382 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:01,383 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:01,383 - DEBUG - ls -l testfile 2025-07-22 17:22:01,781 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:01,781 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:04,782 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:04,783 - DEBUG - ls -l testfile 2025-07-22 17:22:05,115 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:05,115 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:08,116 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:08,116 - DEBUG - ls -l testfile 2025-07-22 17:22:08,434 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:08,434 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:11,435 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:11,436 - DEBUG - ls -l testfile 2025-07-22 17:22:11,765 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:11,765 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:14,766 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:14,766 - DEBUG - ls -l testfile 2025-07-22 17:22:15,111 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:15,111 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:18,112 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:18,112 - DEBUG - ls -l testfile 2025-07-22 17:22:18,442 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:18,442 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:21,443 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:21,443 - DEBUG - ls -l testfile 2025-07-22 17:22:21,780 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:21,780 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:24,780 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:24,781 - DEBUG - ls -l testfile 2025-07-22 17:22:25,163 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:25,163 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:28,164 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:28,164 - DEBUG - ls -l testfile 2025-07-22 17:22:28,528 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:28,528 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:28,528 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:22:28,528 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10002 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:22:28,529 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10002 104.12.205.196 5201 < testfile 2025-07-22 17:22:31,116 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000914118 s, 109 kB/s 2025-07-22 17:22:31,117 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:31,117 - DEBUG - ls -l testfile 2025-07-22 17:22:31,436 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:31,437 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:34,437 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:34,438 - DEBUG - ls -l testfile 2025-07-22 17:22:34,757 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:34,757 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:37,758 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:37,758 - DEBUG - ls -l testfile 2025-07-22 17:22:38,100 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:38,100 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:41,101 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:41,101 - DEBUG - ls -l testfile 2025-07-22 17:22:41,428 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:41,428 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:44,429 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:44,430 - DEBUG - ls -l testfile 2025-07-22 17:22:44,768 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:44,768 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:47,769 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:47,769 - DEBUG - ls -l testfile 2025-07-22 17:22:48,124 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:48,124 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:51,124 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:51,124 - DEBUG - ls -l testfile 2025-07-22 17:22:51,458 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:51,458 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:54,459 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:54,459 - DEBUG - ls -l testfile 2025-07-22 17:22:54,790 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:21 testfile 2025-07-22 17:22:54,790 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:22:57,791 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:57,791 - DEBUG - ls -l testfile 2025-07-22 17:22:58,113 - DEBUG - -rw-r--r-- 1 root root 100 Jul 22 17:22 testfile 2025-07-22 17:22:58,113 - INFO - File transfer verification for file size 100 passed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:22:58,114 - DEBUG - Running remote_cmd, Cmd : echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:22:58,114 - DEBUG - echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:22:58,539 - DEBUG - None 2025-07-22 17:22:58,539 - DEBUG - Running remote_cmd, Cmd : echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:22:58,539 - DEBUG - echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:22:59,041 - DEBUG - None 2025-07-22 17:22:59,041 - INFO - {'echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:22:59,041 - INFO - {'echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:22:59,041 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:22:59,041 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 104.12.205.196, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:22:59,041 - DEBUG - ping -s 56 -c 3 -W 1 104.12.205.196 2025-07-22 17:23:01,496 - DEBUG - PING 104.12.205.196 (104.12.205.196) 56(84) bytes of data. 64 bytes from 104.12.205.196: icmp_req=1 ttl=64 time=4.90 ms 64 bytes from 104.12.205.196: icmp_req=2 ttl=64 time=1.13 ms 64 bytes from 104.12.205.196: icmp_req=3 ttl=64 time=1.15 ms --- 104.12.205.196 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.131/2.396/4.903/1.772 ms 2025-07-22 17:23:01,496 - INFO - Ping to IP 104.12.205.196 from VM ctest-TestMtuProbe-84168175-00542809 passed 2025-07-22 17:23:01,496 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:23:01,496 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:01,497 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile & 2025-07-22 17:23:02,059 - DEBUG - total 132 drwxr-xr-x 11 ubuntu ubuntu 4096 Jul 22 17:23 . drwxr-xr-x 3 root root 4096 Jan 4 2013 .. -rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history -rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout -rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache -rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins -rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla -rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh -rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh -rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0 -rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz -rw------- 1 root root 4695 Dec 17 2017 .viminfo -rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority -rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors 2025-07-22 17:23:02,059 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -ll 5201 > testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:02,059 - DEBUG - nohup nc -4 -q 2 -w 5 -ll 5201 > testfile & 2025-07-22 17:23:02,522 - DEBUG - None 2025-07-22 17:23:02,522 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:23:02,522 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:23:08,057 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000912393 s, 110 kB/s 2025-07-22 17:23:08,057 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:08,057 - DEBUG - ls -l testfile 2025-07-22 17:23:08,409 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:08,409 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:11,410 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:11,410 - DEBUG - ls -l testfile 2025-07-22 17:23:11,749 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:11,749 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:14,750 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:14,750 - DEBUG - ls -l testfile 2025-07-22 17:23:15,085 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:15,085 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:18,086 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:18,086 - DEBUG - ls -l testfile 2025-07-22 17:23:18,415 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:18,415 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:21,415 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:21,415 - DEBUG - ls -l testfile 2025-07-22 17:23:21,765 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:21,765 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:24,766 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:24,767 - DEBUG - ls -l testfile 2025-07-22 17:23:25,097 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:25,097 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:28,098 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:28,098 - DEBUG - ls -l testfile 2025-07-22 17:23:28,505 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:28,505 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:31,505 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:31,506 - DEBUG - ls -l testfile 2025-07-22 17:23:31,835 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:31,835 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:34,836 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:34,836 - DEBUG - ls -l testfile 2025-07-22 17:23:35,174 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:35,174 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:38,175 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:38,175 - DEBUG - ls -l testfile 2025-07-22 17:23:38,530 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:38,530 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:41,531 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:41,531 - DEBUG - ls -l testfile 2025-07-22 17:23:41,871 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:41,871 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:41,871 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:23:41,871 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:23:41,871 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:23:44,416 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.00086134 s, 116 kB/s 2025-07-22 17:23:44,416 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:44,416 - DEBUG - ls -l testfile 2025-07-22 17:23:44,737 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:44,737 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:47,738 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:47,738 - DEBUG - ls -l testfile 2025-07-22 17:23:48,055 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:48,055 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:51,055 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:51,055 - DEBUG - ls -l testfile 2025-07-22 17:23:51,397 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:51,397 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:54,397 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:54,398 - DEBUG - ls -l testfile 2025-07-22 17:23:54,715 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:54,716 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:23:57,716 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:23:57,717 - DEBUG - ls -l testfile 2025-07-22 17:23:58,079 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:23:58,079 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:01,080 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:01,081 - DEBUG - ls -l testfile 2025-07-22 17:24:01,405 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:24:01,405 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:04,406 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:04,406 - DEBUG - ls -l testfile 2025-07-22 17:24:04,750 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:24:04,750 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:07,751 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:07,751 - DEBUG - ls -l testfile 2025-07-22 17:24:08,100 - DEBUG - -rw-r--r-- 1 root root 0 Jul 22 17:23 testfile 2025-07-22 17:24:08,100 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:11,100 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:11,100 - DEBUG - ls -l testfile 2025-07-22 17:24:11,431 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:11,431 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:14,432 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:14,432 - DEBUG - ls -l testfile 2025-07-22 17:24:14,761 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:14,761 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:17,762 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:17,763 - DEBUG - ls -l testfile 2025-07-22 17:24:18,107 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:18,107 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:18,107 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:24:18,107 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:24:18,107 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:24:18,622 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000863961 s, 116 kB/s nc: bind failed: Address already in use 2025-07-22 17:24:18,622 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:24:18,622 - DEBUG - nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:24:19,118 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:24:25,118 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:24:25,119 - DEBUG - nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:24:25,576 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:24:31,577 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:24:31,577 - DEBUG - nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:24:32,056 - DEBUG - nc: bind failed: Address already in use 2025-07-22 17:24:38,056 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:24:38,056 - DEBUG - nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:24:43,515 - DEBUG - None 2025-07-22 17:24:43,516 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:43,516 - DEBUG - ls -l testfile 2025-07-22 17:24:43,874 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:43,874 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:46,875 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:46,876 - DEBUG - ls -l testfile 2025-07-22 17:24:47,203 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:47,204 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:50,204 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:50,205 - DEBUG - ls -l testfile 2025-07-22 17:24:50,537 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:50,537 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:53,538 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:53,538 - DEBUG - ls -l testfile 2025-07-22 17:24:53,874 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:53,874 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:24:56,874 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:24:56,875 - DEBUG - ls -l testfile 2025-07-22 17:24:57,215 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:24:57,216 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:00,217 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:00,217 - DEBUG - ls -l testfile 2025-07-22 17:25:00,565 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:00,565 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:03,566 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:03,567 - DEBUG - ls -l testfile 2025-07-22 17:25:03,907 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:03,907 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:06,907 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:06,908 - DEBUG - ls -l testfile 2025-07-22 17:25:07,253 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:07,253 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:10,254 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:10,254 - DEBUG - ls -l testfile 2025-07-22 17:25:10,588 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:10,588 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:13,588 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:13,588 - DEBUG - ls -l testfile 2025-07-22 17:25:13,917 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:13,917 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:16,918 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:16,919 - DEBUG - ls -l testfile 2025-07-22 17:25:17,238 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:17,238 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:17,238 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:25:17,238 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:25:17,238 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:25:17,771 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000910212 s, 110 kB/s 2025-07-22 17:25:17,772 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:17,772 - DEBUG - ls -l testfile 2025-07-22 17:25:18,124 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:18,124 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:21,125 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:21,126 - DEBUG - ls -l testfile 2025-07-22 17:25:21,455 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:21,456 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:24,457 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:24,457 - DEBUG - ls -l testfile 2025-07-22 17:25:24,790 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:24,790 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:27,791 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:27,791 - DEBUG - ls -l testfile 2025-07-22 17:25:28,133 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:28,133 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:31,134 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:31,134 - DEBUG - ls -l testfile 2025-07-22 17:25:31,474 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:31,474 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:34,475 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:34,475 - DEBUG - ls -l testfile 2025-07-22 17:25:34,818 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:34,819 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:37,819 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:37,819 - DEBUG - ls -l testfile 2025-07-22 17:25:38,174 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:38,175 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:41,175 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:41,176 - DEBUG - ls -l testfile 2025-07-22 17:25:41,520 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:41,520 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:44,521 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:44,521 - DEBUG - ls -l testfile 2025-07-22 17:25:44,861 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:44,861 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:47,862 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:47,862 - DEBUG - ls -l testfile 2025-07-22 17:25:48,222 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:48,222 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:51,222 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:51,222 - DEBUG - ls -l testfile 2025-07-22 17:25:51,549 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:51,550 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:51,550 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:25:51,550 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:25:51,550 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 104.12.205.196 5201 < testfile 2025-07-22 17:25:52,086 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000921843 s, 108 kB/s 2025-07-22 17:25:52,087 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:52,087 - DEBUG - ls -l testfile 2025-07-22 17:25:52,415 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:52,415 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:55,416 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:55,416 - DEBUG - ls -l testfile 2025-07-22 17:25:55,757 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:55,757 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:25:58,758 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:25:58,758 - DEBUG - ls -l testfile 2025-07-22 17:25:59,097 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:25:59,097 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:02,097 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:02,098 - DEBUG - ls -l testfile 2025-07-22 17:26:02,423 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:02,423 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:05,423 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:05,423 - DEBUG - ls -l testfile 2025-07-22 17:26:05,767 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:05,767 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:08,768 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:08,768 - DEBUG - ls -l testfile 2025-07-22 17:26:09,110 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:09,110 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:12,111 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:12,111 - DEBUG - ls -l testfile 2025-07-22 17:26:12,545 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:12,545 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:15,545 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:15,545 - DEBUG - ls -l testfile 2025-07-22 17:26:15,890 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:15,890 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:18,890 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:18,890 - DEBUG - ls -l testfile 2025-07-22 17:26:19,234 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:19,234 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:22,235 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:22,235 - DEBUG - ls -l testfile 2025-07-22 17:26:22,566 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:22,566 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:25,567 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:26:25,567 - DEBUG - ls -l testfile 2025-07-22 17:26:25,911 - DEBUG - -rw-r--r-- 1 root root 64 Jul 22 17:24 testfile 2025-07-22 17:26:25,911 - DEBUG - File size 100 verification failed on the VM, will retry after 3 seconds 2025-07-22 17:26:25,911 - INFO - File transfer verification for file size 100 failed on the VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:26:25,913 - INFO - Deleting VM ctest-TestMtuProbe-84168175-56568640 2025-07-22 17:26:26,011 - INFO - Deleting VM ctest-TestMtuProbe-84168175-00542809 2025-07-22 17:26:26,092 - INFO - Deleting VN ctest-vn-90980520 2025-07-22 17:26:26,204 - DEBUG - VN 7e487ebe-0829-4370-8dc4-8f8e293050a6 still in use: Unable to complete operation on network 7e487ebe-0829-4370-8dc4-8f8e293050a6. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-d6da449c-ab94-4211-9964-13904cc6805c'] 2025-07-22 17:26:26,204 - WARNING - Deleting VN ctest-vn-90980520 failed..Will retry 2025-07-22 17:26:28,547 - DEBUG - Response for deleting network () 2025-07-22 17:26:28,547 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/408d62b0-b9c4-49c0-a68a-92d75d6cb45b 2025-07-22 17:26:28,613 - DEBUG - Response Code: 404 2025-07-22 17:26:28,613 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:26:28,624 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:26:28,653 - DEBUG - Requesting: http://10.0.0.254:8082/project/bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:26:28,758 - INFO - Validated that VN ctest-vn-90980520 is not found in API Server 2025-07-22 17:26:28,770 - DEBUG - VN ctest-vn-90980520 is not present in Agent 10.0.0.19 2025-07-22 17:26:28,781 - DEBUG - VN ctest-vn-90980520 is not present in Agent 10.0.0.27 2025-07-22 17:26:28,781 - INFO - Validated that VN ctest-vn-90980520 is not in any agent 2025-07-22 17:26:28,828 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-90980520 info 2025-07-22 17:26:30,077 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Tue Jul 22 17:26:25 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/tcutils/util.py in wrapper(self=, *func_args=(), **func_kwargs={}) 1039 raise testtools.TestCase.skipException(msg) 1040 1041 return f(self, *func_args, **func_kwargs) 1042 return wrapper 1043 return decorator f = self = func_args = () func_kwargs = {} /contrail-test/scripts/vrouter/test_tcp_mtu_probe.py in test_mtu_probe_inter_node(self=) 84 self.configure_tcp_mtu_probe(client,server,value) 85 self.verify_icmp_traffic(client,server) 86 self.verify_tcp_traffic(client,server, value) 87 return True 88 self = self.verify_tcp_traffic = > client = server = value = 2 /contrail-test/scripts/vrouter/test_tcp_mtu_probe.py in verify_tcp_traffic(self=, client=, server=, value=2) 42 if value is 2: 43 sport += value 44 assert self.send_nc_traffic(client, server, sport, dport, proto), "TCP traffic failed" 45 return True 46 self = self.send_nc_traffic = > client = server = sport = 10003 dport = 5201 proto = 'tcp' AssertionError: TCP traffic failed __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('TCP traffic failed',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/tcutils/util.py", line 1041, in wrapper return f(self, *func_args, **func_kwargs) File "/contrail-test/scripts/vrouter/test_tcp_mtu_probe.py", line 86, in test_mtu_probe_inter_node self.verify_tcp_traffic(client,server, value) File "/contrail-test/scripts/vrouter/test_tcp_mtu_probe.py", line 44, in verify_tcp_traffic assert self.send_nc_traffic(client, server, sport, dport, proto), "TCP traffic failed" AssertionError: TCP traffic failed 2025-07-22 17:26:30,091 - DEBUG - No XMPP flaps were noticed during the test 2025-07-22 17:26:30,091 - INFO -
2025-07-22 17:26:30,091 - INFO - END TEST : test_mtu_probe_inter_node : FAILED[0:09:21]
2025-07-22 17:26:30,091 - INFO - -------------------------------------------------------------------------------- 2025-07-22 17:26:30,102 - INFO - ================================================================================ 2025-07-22 17:26:30,102 - INFO - STARTING TEST : test_mtu_probe_intra_node 2025-07-22 17:26:30,102 - INFO - TEST DESCRIPTION : Description: check traffic configuring different tcp_mtu_probe vlaues on vm MTU probing values: 0, 1, 2 0 - Disabled (default) 1 - Disabled by default, Enabled only when ICMP black hole detected 2 - Always enabled, using the initial MSS value of tcp_base_mss system internally enables bit values for point 1&2 For test purpose, Idea is to check traffic works fine on configuring values Vm's across compute node steps: 1. create 1 VN and launch 2 VMs in same compute node 2. client and server VMs in same compute node 3. configure tcp_mtu_probe to 0, 1, 2 4. send icmp traffic assert if ping fails 5. send tcp traffic assert if tcp traffic fails Pass criteria: 1. If icmp/tcp traffic work fine configuring tcp_mtu_probe values 2025-07-22 17:26:31,402 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-07-22 17:26:31,403 - INFO - Initial checks done. Running the testcase now 2025-07-22 17:26:31,403 - INFO - 2025-07-22 17:26:31,769 - DEBUG - Response for create_network : {'network': {'id': '3db17a96-5a35-4998-a02e-a5d9d986ddb9', 'name': 'ctest-vn-91480462', 'tenant_id': 'bcea95588341461288c40d8752eeb9f2', 'project_id': 'bcea95588341461288c40d8752eeb9f2', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMtuProbe-84168175', 'ctest-vn-91480462'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:26:32,000 - DEBUG - Response for create_subnet : {'subnet': {'id': '7ea3b14e-5491-4251-b1cc-347107989731', 'name': '', 'tenant_id': 'bcea95588341461288c40d8752eeb9f2', 'network_id': '3db17a96-5a35-4998-a02e-a5d9d986ddb9', 'ip_version': 4, 'cidr': '18.236.23.128/26', 'allocation_pools': [{'start': '18.236.23.130', 'end': '18.236.23.190'}], 'gateway_ip': '18.236.23.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '18.236.23.130', 'tags': [], 'project_id': 'bcea95588341461288c40d8752eeb9f2'}} 2025-07-22 17:26:32,028 - INFO - Created VN ctest-vn-91480462 2025-07-22 17:26:32,046 - DEBUG - VN ctest-vn-91480462 UUID is 3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,132 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:26:32,140 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:26:32,156 - DEBUG - Requesting: http://10.0.0.254:8082/project/bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:26:32,251 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,265 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,276 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5bc3acb-c1b2-4513-9139-474efe0e4e1c 2025-07-22 17:26:32,285 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5bc3acb-c1b2-4513-9139-474efe0e4e1c 2025-07-22 17:26:32,293 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/e85c2249-5990-4062-8f15-c15c0bd2035e 2025-07-22 17:26:32,303 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:26:32,303 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,315 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5bc3acb-c1b2-4513-9139-474efe0e4e1c 2025-07-22 17:26:32,333 - INFO - Verified VN network id 12 for VN 3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,333 - INFO - Verifications in API Server for VN ctest-vn-91480462 passed 2025-07-22 17:26:32,333 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,344 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5bc3acb-c1b2-4513-9139-474efe0e4e1c 2025-07-22 17:26:32,352 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5bc3acb-c1b2-4513-9139-474efe0e4e1c 2025-07-22 17:26:32,361 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/e85c2249-5990-4062-8f15-c15c0bd2035e 2025-07-22 17:26:32,374 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestMtuProbe-84168175', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4445469093368514968', 'uuid-lslong': '11542345250073140665'}, 'enable': 'true', 'created': '2025-07-22T17:26:31', 'last-modified': '2025-07-22T17:26:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.436351'} 2025-07-22 17:26:32,382 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:26:32,388 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestMtuProbe-84168175', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4445469093368514968', 'uuid-lslong': '11542345250073140665'}, 'enable': 'true', 'created': '2025-07-22T17:26:31', 'last-modified': '2025-07-22T17:26:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.453806'} 2025-07-22 17:26:32,396 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:26:32,403 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestMtuProbe-84168175', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4445469093368514968', 'uuid-lslong': '11542345250073140665'}, 'enable': 'true', 'created': '2025-07-22T17:26:31', 'last-modified': '2025-07-22T17:26:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.471589'} 2025-07-22 17:26:32,410 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:26:32,410 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-91480462 passed 2025-07-22 17:26:32,410 - DEBUG - ====Verifying policy data for ctest-vn-91480462 in API_Server ====== 2025-07-22 17:26:32,410 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:26:32,417 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:26:32,430 - DEBUG - Requesting: http://10.0.0.254:8082/project/bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:26:32,536 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3db17a96-5a35-4998-a02e-a5d9d986ddb9 2025-07-22 17:26:32,552 - DEBUG - =>VN ctest-vn-91480462 has no policy to be verified 2025-07-22 17:26:32,552 - DEBUG - Verifying the vn in opserver 2025-07-22 17:26:32,552 - DEBUG - Verifying the default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 virtual network link through opserver 10.0.0.254 2025-07-22 17:26:32,552 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:26:32,624 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462?flat'} 2025-07-22 17:26:32,624 - DEBUG - Verifying the default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 virtual network link through opserver 10.0.0.50 2025-07-22 17:26:32,624 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:26:32,683 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462?flat'} 2025-07-22 17:26:32,683 - DEBUG - Verifying the default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 virtual network link through opserver 10.0.0.23 2025-07-22 17:26:32,683 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:26:32,741 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462?flat'} 2025-07-22 17:26:32,741 - INFO - Validated that VN default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is found in opserver 2025-07-22 17:26:32,766 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 17:26:32,778 - DEBUG - VRF ids for VN ctest-vn-91480462: {} 2025-07-22 17:26:32,778 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:26:32,778 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:26:33,994 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 17:26:33,995 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:26:35,480 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 17:26:35,480 - INFO - Waiting for VM ctest-TestMtuProbe-84168175-96943673 to be up.. 2025-07-22 17:26:35,592 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-07-22 17:26:40,685 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:26:40,685 - INFO - VM name : ctest-TestMtuProbe-84168175-96943673 2025-07-22 17:26:40,766 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 ID is db1a5c4c-fc2d-43bc-bd84-d29db6bbdde2 2025-07-22 17:26:40,766 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 17:26:40,845 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/db1a5c4c-fc2d-43bc-bd84-d29db6bbdde2 2025-07-22 17:26:40,881 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/52eb4971-4e91-4d2f-988b-5baae269b6c1 2025-07-22 17:26:44,088 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1002ms') 2025-07-22 17:26:44,088 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:26:44,105 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:26:48,185 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 17:26:48,186 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:26:48,201 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:26:52,286 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1026ms') 2025-07-22 17:26:52,286 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:26:52,303 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:26:56,383 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-07-22 17:26:56,384 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:26:56,400 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:27:00,480 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-07-22 17:27:00,480 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:27:00,496 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:27:04,575 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-07-22 17:27:04,576 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:27:04,594 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:27:08,670 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-07-22 17:27:08,671 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:27:08,689 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:27:12,767 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-07-22 17:27:12,768 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:27:12,784 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:27:16,861 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-07-22 17:27:16,861 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 failed! 2025-07-22 17:27:16,879 - DEBUG - Gateway for vn default-domain:ctest-TestMtuProbe-84168175:ctest-vn-91480462 is 18.236.23.129 and allocation pool is NOT set 2025-07-22 17:27:20,952 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=14.2 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1013ms\r\nrtt min/avg/max/mdev = 14.152/14.152/14.152/0.000 ms') 2025-07-22 17:27:20,952 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestMtuProbe-84168175-96943673 passed 2025-07-22 17:27:21,070 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:27:21,070 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-96943673, IP 18.236.23.131, Port 22 2025-07-22 17:27:21,140 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:27:21,230 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:27:26,232 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:27:26,232 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-96943673, IP 18.236.23.131, Port 22 2025-07-22 17:27:26,293 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:27:26,378 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:27:31,379 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:27:31,379 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-96943673, IP 18.236.23.131, Port 22 2025-07-22 17:27:31,437 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:27:31,524 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:27:36,525 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:27:36,525 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-96943673, IP 18.236.23.131, Port 22 2025-07-22 17:27:36,592 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:27:36,681 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:27:41,682 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:27:41,683 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-96943673, IP 18.236.23.131, Port 22 2025-07-22 17:27:41,855 - DEBUG - VM ctest-TestMtuProbe-84168175-96943673 is ready for SSH connections 2025-07-22 17:27:41,856 - INFO - Waiting for VM ctest-TestMtuProbe-84168175-00367887 to be up.. 2025-07-22 17:27:41,940 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:27:41,941 - INFO - VM name : ctest-TestMtuProbe-84168175-00367887 2025-07-22 17:27:42,024 - DEBUG - VM ctest-TestMtuProbe-84168175-00367887 ID is bed8e2b5-2a29-423f-a2e5-294e341bcd8f 2025-07-22 17:27:42,024 - DEBUG - VM ctest-TestMtuProbe-84168175-00367887 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 17:27:42,107 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/bed8e2b5-2a29-423f-a2e5-294e341bcd8f 2025-07-22 17:27:42,119 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/3f8f2690-9a7d-4b26-8e91-14d86e16ab2a 2025-07-22 17:27:43,335 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=1.78 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=2.75 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.784/2.269/2.754/0.485 ms') 2025-07-22 17:27:43,336 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestMtuProbe-84168175-00367887 passed 2025-07-22 17:27:43,451 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:27:43,451 - DEBUG - Waiting to SSH to VM ctest-TestMtuProbe-84168175-00367887, IP 18.236.23.132, Port 22 2025-07-22 17:27:43,619 - DEBUG - VM ctest-TestMtuProbe-84168175-00367887 is ready for SSH connections 2025-07-22 17:27:43,619 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:27:43,619 - DEBUG - echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:28:05,464 - DEBUG - None 2025-07-22 17:28:05,464 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:05,464 - DEBUG - echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:28:41,525 - DEBUG - None 2025-07-22 17:28:41,526 - INFO - {'echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:28:41,526 - INFO - {'echo 0 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:28:41,526 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:28:41,526 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 18.236.23.132, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:41,526 - DEBUG - ping -s 56 -c 3 -W 1 18.236.23.132 2025-07-22 17:28:43,994 - DEBUG - PING 18.236.23.132 (18.236.23.132) 56(84) bytes of data. 64 bytes from 18.236.23.132: icmp_req=1 ttl=64 time=9.35 ms 64 bytes from 18.236.23.132: icmp_req=2 ttl=64 time=1.03 ms 64 bytes from 18.236.23.132: icmp_req=3 ttl=64 time=0.758 ms --- 18.236.23.132 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.758/3.716/9.358/3.991 ms 2025-07-22 17:28:43,994 - INFO - Ping to IP 18.236.23.132 from VM ctest-TestMtuProbe-84168175-96943673 passed 2025-07-22 17:28:43,994 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:28:43,994 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:43,994 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile & 2025-07-22 17:28:44,561 - DEBUG - total 132 drwxr-xr-x 11 ubuntu ubuntu 4096 Dec 29 2017 . drwxr-xr-x 3 root root 4096 Jan 4 2013 .. -rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history -rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout -rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache -rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins -rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla -rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh -rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh -rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0 -rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz -rw------- 1 root root 4695 Dec 17 2017 .viminfo -rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority -rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors 2025-07-22 17:28:44,561 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -ll 5201 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:44,562 - DEBUG - nohup nc -4 -q 2 -w 5 -ll 5201 > testfile & 2025-07-22 17:28:44,991 - DEBUG - None 2025-07-22 17:28:44,991 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 18.236.23.132 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:44,991 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10001 18.236.23.132 5201 < testfile 2025-07-22 17:28:47,535 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000842826 s, 119 kB/s 2025-07-22 17:28:47,535 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:47,535 - DEBUG - ls -l testfile 2025-07-22 17:28:47,860 - DEBUG - -rw-r--r-- 1 root root 100 Jul 22 17:28 testfile 2025-07-22 17:28:47,860 - INFO - File transfer verification for file size 100 passed on the VM ctest-TestMtuProbe-84168175-00367887 2025-07-22 17:28:47,860 - DEBUG - Running remote_cmd, Cmd : echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:47,861 - DEBUG - echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:28:48,264 - DEBUG - None 2025-07-22 17:28:48,264 - DEBUG - Running remote_cmd, Cmd : echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:48,264 - DEBUG - echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:28:48,715 - DEBUG - None 2025-07-22 17:28:48,716 - INFO - {'echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:28:48,716 - INFO - {'echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:28:48,716 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:28:48,716 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 18.236.23.132, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:48,716 - DEBUG - ping -s 56 -c 3 -W 1 18.236.23.132 2025-07-22 17:28:51,179 - DEBUG - PING 18.236.23.132 (18.236.23.132) 56(84) bytes of data. 64 bytes from 18.236.23.132: icmp_req=1 ttl=64 time=1.38 ms 64 bytes from 18.236.23.132: icmp_req=2 ttl=64 time=0.860 ms 64 bytes from 18.236.23.132: icmp_req=3 ttl=64 time=0.914 ms --- 18.236.23.132 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.860/1.051/1.380/0.234 ms 2025-07-22 17:28:51,180 - INFO - Ping to IP 18.236.23.132 from VM ctest-TestMtuProbe-84168175-96943673 passed 2025-07-22 17:28:51,180 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:28:51,180 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:51,180 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile & 2025-07-22 17:28:51,715 - DEBUG - total 132 drwxr-xr-x 11 ubuntu ubuntu 4096 Jul 22 17:28 . drwxr-xr-x 3 root root 4096 Jan 4 2013 .. -rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history -rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout -rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache -rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins -rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla -rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh -rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh -rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0 -rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz -rw------- 1 root root 4695 Dec 17 2017 .viminfo -rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority -rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors 2025-07-22 17:28:51,715 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -ll 5201 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:51,715 - DEBUG - nohup nc -4 -q 2 -w 5 -ll 5201 > testfile & 2025-07-22 17:28:52,171 - DEBUG - None 2025-07-22 17:28:52,171 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10002 18.236.23.132 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:52,171 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10002 18.236.23.132 5201 < testfile 2025-07-22 17:28:54,708 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.00204924 s, 48.8 kB/s 2025-07-22 17:28:54,709 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:54,709 - DEBUG - ls -l testfile 2025-07-22 17:28:55,031 - DEBUG - -rw-r--r-- 1 root root 100 Jul 22 17:28 testfile 2025-07-22 17:28:55,031 - INFO - File transfer verification for file size 100 passed on the VM ctest-TestMtuProbe-84168175-00367887 2025-07-22 17:28:55,031 - DEBUG - Running remote_cmd, Cmd : echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:55,031 - DEBUG - echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:28:55,443 - DEBUG - None 2025-07-22 17:28:55,443 - DEBUG - Running remote_cmd, Cmd : echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:55,443 - DEBUG - echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing 2025-07-22 17:28:55,860 - DEBUG - None 2025-07-22 17:28:55,860 - INFO - {'echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:28:55,860 - INFO - {'echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing': None} 2025-07-22 17:28:55,860 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:28:55,860 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 18.236.23.132, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:55,861 - DEBUG - ping -s 56 -c 3 -W 1 18.236.23.132 2025-07-22 17:28:58,310 - DEBUG - PING 18.236.23.132 (18.236.23.132) 56(84) bytes of data. 64 bytes from 18.236.23.132: icmp_req=1 ttl=64 time=2.31 ms 64 bytes from 18.236.23.132: icmp_req=2 ttl=64 time=0.790 ms 64 bytes from 18.236.23.132: icmp_req=3 ttl=64 time=0.671 ms --- 18.236.23.132 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 0.671/1.258/2.314/0.748 ms 2025-07-22 17:28:58,310 - INFO - Ping to IP 18.236.23.132 from VM ctest-TestMtuProbe-84168175-96943673 passed 2025-07-22 17:28:58,310 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:28:58,310 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:58,310 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -ll -p 5201 > testfile & 2025-07-22 17:28:58,849 - DEBUG - total 132 drwxr-xr-x 11 ubuntu ubuntu 4096 Jul 22 17:28 . drwxr-xr-x 3 root root 4096 Jan 4 2013 .. -rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history -rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout -rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache -rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins -rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla -rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh -rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh -rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0 -rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz -rw------- 1 root root 4695 Dec 17 2017 .viminfo -rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority -rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors 2025-07-22 17:28:58,849 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -ll 5201 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:58,850 - DEBUG - nohup nc -4 -q 2 -w 5 -ll 5201 > testfile & 2025-07-22 17:28:59,291 - DEBUG - None 2025-07-22 17:28:59,291 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 18.236.23.132 5201 < testfile, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:28:59,291 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -p 10003 18.236.23.132 5201 < testfile 2025-07-22 17:29:01,827 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000928743 s, 108 kB/s 2025-07-22 17:29:01,827 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:29:01,827 - DEBUG - ls -l testfile 2025-07-22 17:29:02,148 - DEBUG - -rw-r--r-- 1 root root 100 Jul 22 17:28 testfile 2025-07-22 17:29:02,148 - INFO - File transfer verification for file size 100 passed on the VM ctest-TestMtuProbe-84168175-00367887 2025-07-22 17:29:02,148 - INFO - Deleting VM ctest-TestMtuProbe-84168175-00367887 2025-07-22 17:29:02,266 - INFO - Deleting VM ctest-TestMtuProbe-84168175-96943673 2025-07-22 17:29:02,372 - INFO - Deleting VN ctest-vn-91480462 2025-07-22 17:29:02,419 - DEBUG - VN 3db17a96-5a35-4998-a02e-a5d9d986ddb9 still in use: Unable to complete operation on network 3db17a96-5a35-4998-a02e-a5d9d986ddb9. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-f7fd5595-ca5f-44b1-a05c-9e7a3c90aafe'] 2025-07-22 17:29:02,419 - WARNING - Deleting VN ctest-vn-91480462 failed..Will retry 2025-07-22 17:29:04,635 - DEBUG - Response for deleting network () 2025-07-22 17:29:04,635 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5bc3acb-c1b2-4513-9139-474efe0e4e1c 2025-07-22 17:29:04,645 - DEBUG - Response Code: 404 2025-07-22 17:29:04,645 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:29:04,653 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:29:04,710 - DEBUG - Requesting: http://10.0.0.254:8082/project/bcea9558-8341-4612-88c4-0d8752eeb9f2 2025-07-22 17:29:04,805 - INFO - Validated that VN ctest-vn-91480462 is not found in API Server 2025-07-22 17:29:04,821 - DEBUG - VN ctest-vn-91480462 is not present in Agent 10.0.0.19 2025-07-22 17:29:04,833 - DEBUG - VN ctest-vn-91480462 is not present in Agent 10.0.0.27 2025-07-22 17:29:04,833 - INFO - Validated that VN ctest-vn-91480462 is not in any agent 2025-07-22 17:29:04,867 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-91480462 info 2025-07-22 17:29:06,143 - DEBUG - No XMPP flaps were noticed during the test
2025-07-22 17:29:06,143 - INFO - END TEST : test_mtu_probe_intra_node : PASSED[0:02:36]
2025-07-22 17:29:06,143 - INFO - -------------------------------------------------------------------------------- 2025-07-22 17:29:06,928 - INFO - Deleted project: ctest-TestMtuProbe-84168175, ID : bcea9558-8341-4612-88c4-0d8752eeb9f2