2025-08-04 02:37:52,266 - INFO - Domain Default found not creating 2025-08-04 02:37:52,431 - INFO - Project ctest-TestDSNAT-34673403 not found, creating it 2025-08-04 02:37:53,325 - INFO - Created Project:ctest-TestDSNAT-34673403, ID : 37766e41-2c6e-40af-8a2b-d98eaea73404 2025-08-04 02:37:55,095 - INFO - ================================================================================ 2025-08-04 02:37:55,095 - INFO - STARTING TEST : test_dsnat_basic 2025-08-04 02:37:55,095 - INFO - TEST DESCRIPTION : create a VN and enable fabric SNAT launch two VMs in that VN verify ping between the VN and ping to the external IP disable fabric SNAT verify that the ping the external IP fails 2025-08-04 02:37:56,391 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with 2025-08-04 02:37:56,391 - INFO - Initial checks done. Running the testcase now 2025-08-04 02:37:56,391 - INFO - 2025-08-04 02:37:56,391 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance 2025-08-04 02:37:57,122 - DEBUG - Response for create_network : {'network': {'id': '15c8e69e-e22e-423c-a45d-55dab683dcb3', 'name': 'ctest-dsnat_vn-82627359', 'tenant_id': '37766e412c6e40af8a2bd98eaea73404', 'project_id': '37766e412c6e40af8a2bd98eaea73404', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-34673403', 'ctest-dsnat_vn-82627359'], 'port_security_enabled': True, 'description': ''}} 2025-08-04 02:37:57,445 - DEBUG - Response for create_subnet : {'subnet': {'id': '12a1bf4d-365d-4032-85ce-a3406e5e5124', 'name': '', 'tenant_id': '37766e412c6e40af8a2bd98eaea73404', 'network_id': '15c8e69e-e22e-423c-a45d-55dab683dcb3', 'ip_version': 4, 'cidr': '121.130.199.64/26', 'allocation_pools': [{'start': '121.130.199.66', 'end': '121.130.199.126'}], 'gateway_ip': '121.130.199.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '121.130.199.66', 'tags': [], 'project_id': '37766e412c6e40af8a2bd98eaea73404'}} 2025-08-04 02:37:57,476 - INFO - Created VN ctest-dsnat_vn-82627359 2025-08-04 02:37:57,496 - DEBUG - VN ctest-dsnat_vn-82627359 UUID is 15c8e69e-e22e-423c-a45d-55dab683dcb3 2025-08-04 02:37:57,711 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/15c8e69e-e22e-423c-a45d-55dab683dcb3 2025-08-04 02:37:58,036 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/15c8e69e-e22e-423c-a45d-55dab683dcb3 2025-08-04 02:37:58,083 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/06f973cb-0b74-4af9-966c-ccfb35c55a33 2025-08-04 02:37:58,240 - DEBUG - Services list from nova: [, , , ] 2025-08-04 02:37:58,270 - DEBUG - Image by name ubuntu not found 2025-08-04 02:37:58,270 - DEBUG - Installing image ubuntu 2025-08-04 02:37:58,270 - DEBUG - Image info {'name': 'ubuntu.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-08-04 02:37:58,271 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/converts//ubuntu.vmdk.gz 2025-08-04 02:37:58,271 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/converts//ubuntu.vmdk.gz 2025-08-04 02:37:58,286 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/converts//ubuntu.vmdk.gz -O /contrail-test/images/ubuntu.vmdk.gz 2025-08-04 02:37:58,324 - DEBUG - Image has been downloaded to /contrail-test/images/ubuntu.vmdk.gz 2025-08-04 02:37:58,325 - DEBUG - Unzip image 2025-08-04 02:37:58,325 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu.vmdk.gz 2025-08-04 02:37:58,338 - INFO - image /contrail-test/images/ubuntu.vmdk is not found 2025-08-04 02:37:58,339 - INFO - Deleting VN ctest-dsnat_vn-82627359 2025-08-04 02:37:58,558 - DEBUG - Response for deleting network () 2025-08-04 02:37:59,865 - DEBUG - No XMPP flaps were noticed during the test 2025-08-04 02:37:59,865 - INFO -
2025-08-04 02:37:59,865 - INFO - END TEST : test_dsnat_basic : SKIPPED[0:00:04]
2025-08-04 02:37:59,865 - INFO - -------------------------------------------------------------------------------- 2025-08-04 02:37:59,870 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3750-2', 'an-jenkins-deploy-platform-ansible-os-3750-1']} 2025-08-04 02:37:59,870 - INFO - ================================================================================ 2025-08-04 02:37:59,870 - INFO - STARTING TEST : test_dsnat_bug_1749695 2025-08-04 02:37:59,870 - INFO - TEST DESCRIPTION : Testcase to verify the bug 1749695 create test VN , associate a policy to allow all traffic between VN and ip fabric associate the policy to the test VN and fabric VN set the test VN fabric mode as l3 verify the ping succeeds for intra VN and to the fabric IP 2025-08-04 02:38:01,168 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with 2025-08-04 02:38:01,168 - INFO - Initial checks done. Running the testcase now 2025-08-04 02:38:01,168 - INFO - 2025-08-04 02:38:01,168 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance 2025-08-04 02:38:01,487 - DEBUG - Response for create_network : {'network': {'id': '47a92a32-aad3-402b-81bd-801c1975350f', 'name': 'ctest-dsnat_vn-00848659', 'tenant_id': '37766e412c6e40af8a2bd98eaea73404', 'project_id': '37766e412c6e40af8a2bd98eaea73404', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-34673403', 'ctest-dsnat_vn-00848659'], 'port_security_enabled': True, 'description': ''}} 2025-08-04 02:38:01,782 - DEBUG - Response for create_subnet : {'subnet': {'id': '0ce4d734-d30d-42ea-9302-a58d4642a5ef', 'name': '', 'tenant_id': '37766e412c6e40af8a2bd98eaea73404', 'network_id': '47a92a32-aad3-402b-81bd-801c1975350f', 'ip_version': 4, 'cidr': '1.140.120.64/26', 'allocation_pools': [{'start': '1.140.120.66', 'end': '1.140.120.126'}], 'gateway_ip': '1.140.120.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '1.140.120.66', 'tags': [], 'project_id': '37766e412c6e40af8a2bd98eaea73404'}} 2025-08-04 02:38:01,811 - INFO - Created VN ctest-dsnat_vn-00848659 2025-08-04 02:38:01,829 - DEBUG - VN ctest-dsnat_vn-00848659 UUID is 47a92a32-aad3-402b-81bd-801c1975350f 2025-08-04 02:38:02,088 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/47a92a32-aad3-402b-81bd-801c1975350f 2025-08-04 02:38:02,131 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/5b540486-4dd5-41eb-8836-0b64e9b3bd27 2025-08-04 02:38:02,272 - DEBUG - Image by name ubuntu-traffic not found 2025-08-04 02:38:02,272 - DEBUG - Installing image ubuntu-traffic 2025-08-04 02:38:02,272 - DEBUG - Image info {'name': 'ubuntu-traffic.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_small', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'ubuntu-traffic-docker', 'vmdk': 'ubuntu-traffic-disk1.vmdk', 'shrinked_vmdk': "'no'"} 2025-08-04 02:38:02,273 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/converts//ubuntu-traffic.vmdk.gz 2025-08-04 02:38:02,273 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/converts//ubuntu-traffic.vmdk.gz 2025-08-04 02:38:02,288 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/converts//ubuntu-traffic.vmdk.gz -O /contrail-test/images/ubuntu-traffic.vmdk.gz 2025-08-04 02:38:07,489 - DEBUG - Image has been downloaded to /contrail-test/images/ubuntu-traffic.vmdk.gz 2025-08-04 02:38:07,490 - DEBUG - Unzip image 2025-08-04 02:38:07,491 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu-traffic.vmdk.gz 2025-08-04 02:38:26,339 - DEBUG - Try to create image in user context 2025-08-04 02:38:38,245 - DEBUG - Image has been created in user context 2025-08-04 02:38:38,245 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: rm -f /contrail-test/images/ubuntu-traffic.vmdk 2025-08-04 02:38:38,470 - DEBUG - Downloaded image has been removed 2025-08-04 02:38:38,486 - DEBUG - Uploaded image status = active 2025-08-04 02:38:39,685 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3750-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3750-1) 2025-08-04 02:38:41,313 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3750-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3750-2) 2025-08-04 02:38:41,314 - INFO - Waiting for VM ctest-TestDSNAT-34673403-09016583 to be up.. 2025-08-04 02:38:41,477 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:38:46,603 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:38:51,727 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:38:56,821 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:01,927 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:07,019 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:12,137 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:17,225 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:22,327 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:27,423 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:32,546 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 02:39:37,631 - DEBUG - VM is in ACTIVE state now 2025-08-04 02:39:37,632 - INFO - VM name : ctest-TestDSNAT-34673403-09016583 2025-08-04 02:39:37,728 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 ID is fb9326f8-8f78-4b1d-8dee-c39684efdc5d 2025-08-04 02:39:37,766 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 launched on Node an-jenkins-deploy-platform-ansible-os-3750-1 2025-08-04 02:39:37,860 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine/fb9326f8-8f78-4b1d-8dee-c39684efdc5d 2025-08-04 02:39:38,171 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine/fb9326f8-8f78-4b1d-8dee-c39684efdc5d 2025-08-04 02:39:38,210 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine-interface/6950f946-55a1-45ad-88e3-9a82ac73937d 2025-08-04 02:39:41,453 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-08-04 02:39:41,453 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:39:41,472 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:39:45,556 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-08-04 02:39:45,556 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:39:45,575 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:39:49,644 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1005ms') 2025-08-04 02:39:49,644 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:39:49,661 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:39:53,737 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-08-04 02:39:53,737 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:39:53,755 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:39:57,843 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-08-04 02:39:57,843 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:39:57,860 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:40:01,935 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-08-04 02:40:01,935 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:40:01,960 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:40:06,060 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1024ms') 2025-08-04 02:40:06,060 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:40:06,080 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:40:10,152 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-08-04 02:40:10,152 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 failed! 2025-08-04 02:40:10,170 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659 is 1.140.120.65 and allocation pool is NOT set 2025-08-04 02:40:14,255 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=7.72 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1017ms\r\nrtt min/avg/max/mdev = 7.718/7.718/7.718/0.000 ms') 2025-08-04 02:40:14,256 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-09016583 passed 2025-08-04 02:40:14,377 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:40:14,377 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-34673403-09016583, IP 1.140.120.67, Port 22 2025-08-04 02:40:14,444 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 02:40:14,552 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 02:40:19,553 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:40:19,553 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-34673403-09016583, IP 1.140.120.67, Port 22 2025-08-04 02:40:19,621 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 02:40:19,718 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 02:40:24,719 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:40:24,719 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-34673403-09016583, IP 1.140.120.67, Port 22 2025-08-04 02:40:24,787 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 02:40:24,880 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 02:40:29,881 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:40:29,881 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-34673403-09016583, IP 1.140.120.67, Port 22 2025-08-04 02:40:29,939 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 02:40:30,078 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 02:40:35,079 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:40:35,079 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-34673403-09016583, IP 1.140.120.67, Port 22 2025-08-04 02:40:35,248 - DEBUG - VM ctest-TestDSNAT-34673403-09016583 is ready for SSH connections 2025-08-04 02:40:35,248 - INFO - Waiting for VM ctest-TestDSNAT-34673403-36967355 to be up.. 2025-08-04 02:40:35,337 - DEBUG - VM is in ACTIVE state now 2025-08-04 02:40:35,337 - INFO - VM name : ctest-TestDSNAT-34673403-36967355 2025-08-04 02:40:35,428 - DEBUG - VM ctest-TestDSNAT-34673403-36967355 ID is 1e21f4fa-d755-45fe-8b01-b63f835d6795 2025-08-04 02:40:35,428 - DEBUG - VM ctest-TestDSNAT-34673403-36967355 launched on Node an-jenkins-deploy-platform-ansible-os-3750-2 2025-08-04 02:40:35,514 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine/1e21f4fa-d755-45fe-8b01-b63f835d6795 2025-08-04 02:40:35,527 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine-interface/3520a6cd-f22a-40cc-ac3e-f7f34c1a46db 2025-08-04 02:40:36,742 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=2.38 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.745 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.745/1.562/2.380/0.817 ms') 2025-08-04 02:40:36,742 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-34673403-36967355 passed 2025-08-04 02:40:36,870 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:40:36,871 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-34673403-36967355, IP 1.140.120.68, Port 22 2025-08-04 02:40:37,043 - DEBUG - VM ctest-TestDSNAT-34673403-36967355 is ready for SSH connections 2025-08-04 02:40:37,208 - DEBUG - Fetched VN: default-domain:default-project:ip-fabric(f4d5f310-d9e7-46ad-9f1b-5dfe7a69b4e4) with subnets None 2025-08-04 02:40:37,260 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestDSNAT-34673403:ctest-dsnat_vn-00848659, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:default-project:ip-fabric, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-08-04 02:40:37,608 - INFO - Attaching policy ctest-test-dsnat-66609361 to vn ctest-dsnat_vn-00848659 2025-08-04 02:40:38,176 - INFO - Attaching policy ctest-test-dsnat-66609361 to vn ip-fabric 2025-08-04 02:40:38,482 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.193, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 02:40:38,482 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.193 2025-08-04 02:41:00,663 - DEBUG - PING 10.20.0.193 (10.20.0.193) 56(84) bytes of data. 64 bytes from 10.20.0.193: icmp_req=1 ttl=62 time=3.77 ms 64 bytes from 10.20.0.193: icmp_req=2 ttl=62 time=1.07 ms 64 bytes from 10.20.0.193: icmp_req=3 ttl=62 time=0.889 ms --- 10.20.0.193 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.889/1.912/3.778/1.321 ms 2025-08-04 02:41:00,663 - INFO - Ping to IP 10.20.0.193 from VM ctest-TestDSNAT-34673403-09016583 passed 2025-08-04 02:41:00,812 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.193, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 02:41:00,812 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.193 2025-08-04 02:41:04,832 - DEBUG - PING 10.20.0.193 (10.20.0.193) 56(84) bytes of data. --- 10.20.0.193 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms 2025-08-04 02:41:04,832 - WARNING - Ping to IP 10.20.0.193 from VM ctest-TestDSNAT-34673403-09016583 failed 2025-08-04 02:41:05,834 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.193, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 02:41:05,834 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.193 2025-08-04 02:41:09,331 - DEBUG - PING 10.20.0.193 (10.20.0.193) 56(84) bytes of data. --- 10.20.0.193 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2002ms 2025-08-04 02:41:09,331 - WARNING - Ping to IP 10.20.0.193 from VM ctest-TestDSNAT-34673403-09016583 failed 2025-08-04 02:41:10,332 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.193, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 02:41:10,333 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.193 2025-08-04 02:41:13,885 - DEBUG - PING 10.20.0.193 (10.20.0.193) 56(84) bytes of data. --- 10.20.0.193 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms 2025-08-04 02:41:13,885 - WARNING - Ping to IP 10.20.0.193 from VM ctest-TestDSNAT-34673403-09016583 failed 2025-08-04 02:41:14,886 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.193, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 02:41:14,886 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.193 2025-08-04 02:41:17,373 - DEBUG - PING 10.20.0.193 (10.20.0.193) 56(84) bytes of data. 64 bytes from 10.20.0.193: icmp_req=1 ttl=62 time=2.98 ms 64 bytes from 10.20.0.193: icmp_req=2 ttl=62 time=0.989 ms 64 bytes from 10.20.0.193: icmp_req=3 ttl=62 time=0.732 ms --- 10.20.0.193 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 0.732/1.567/2.980/1.004 ms 2025-08-04 02:41:17,373 - INFO - Ping to IP 10.20.0.193 from VM ctest-TestDSNAT-34673403-09016583 passed 2025-08-04 02:41:17,373 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 02:41:17,373 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 1.140.120.68, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 02:41:17,373 - DEBUG - ping -s 56 -c 3 -W 1 1.140.120.68 2025-08-04 02:41:19,900 - DEBUG - PING 1.140.120.68 (1.140.120.68) 56(84) bytes of data. 64 bytes from 1.140.120.68: icmp_req=1 ttl=63 time=10.7 ms 64 bytes from 1.140.120.68: icmp_req=2 ttl=63 time=0.920 ms 64 bytes from 1.140.120.68: icmp_req=3 ttl=63 time=1.16 ms --- 1.140.120.68 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 0.920/4.261/10.701/4.555 ms 2025-08-04 02:41:19,900 - INFO - Ping to IP 1.140.120.68 from VM ctest-TestDSNAT-34673403-09016583 passed 2025-08-04 02:41:19,900 - INFO - Dettaching policy ctest-test-dsnat-66609361 from vn ip-fabric 2025-08-04 02:41:20,117 - INFO - Dettaching policy ctest-test-dsnat-66609361 from vn ctest-dsnat_vn-00848659 2025-08-04 02:41:20,411 - INFO - Deleted policy ctest-test-dsnat-66609361 2025-08-04 02:41:20,411 - INFO - Deleting VM ctest-TestDSNAT-34673403-36967355 2025-08-04 02:41:20,509 - INFO - Deleting VM ctest-TestDSNAT-34673403-09016583 2025-08-04 02:41:20,602 - INFO - Deleting VN ctest-dsnat_vn-00848659 2025-08-04 02:41:20,660 - DEBUG - VN 47a92a32-aad3-402b-81bd-801c1975350f still in use: Unable to complete operation on network 47a92a32-aad3-402b-81bd-801c1975350f. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-1c645f9b-1e25-4b2c-9c62-7e0565ffc0c0'] 2025-08-04 02:41:20,660 - WARNING - Deleting VN ctest-dsnat_vn-00848659 failed..Will retry 2025-08-04 02:41:22,914 - DEBUG - Response for deleting network () 2025-08-04 02:41:24,248 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:41:24,248 - INFO - END TEST : test_dsnat_bug_1749695 : PASSED[0:03:25]
2025-08-04 02:41:24,248 - INFO - -------------------------------------------------------------------------------- 2025-08-04 02:41:25,020 - INFO - Deleted project: ctest-TestDSNAT-34673403, ID : 37766e41-2c6e-40af-8a2b-d98eaea73404 2025-08-04 04:13:09,389 - INFO - Domain Default found not creating 2025-08-04 04:13:10,706 - INFO - Project ctest-TestDSNAT-08888997 not found, creating it 2025-08-04 04:13:12,313 - INFO - Created Project:ctest-TestDSNAT-08888997, ID : 6cad61b2-bda1-4cad-a50a-1d7a8b122a90 2025-08-04 04:13:13,859 - DEBUG - Services list from nova: [, , , ] 2025-08-04 04:13:13,860 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3750-2', 'an-jenkins-deploy-platform-ansible-os-3750-1']} 2025-08-04 04:13:13,860 - INFO - ================================================================================ 2025-08-04 04:13:13,860 - INFO - STARTING TEST : test_dsnat_tcp_basic 2025-08-04 04:13:13,860 - INFO - TEST DESCRIPTION : Configure TCP port translation pool send TCP traffic from the VM to the fabric IP Verify the Port address translation happened and the port being used in the range of configured 2025-08-04 04:13:15,144 - DEBUG - Skipping xmpp flap check 2025-08-04 04:13:15,144 - INFO - Initial checks done. Running the testcase now 2025-08-04 04:13:15,144 - INFO - 2025-08-04 04:13:15,615 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance 2025-08-04 04:13:16,015 - DEBUG - Response for create_network : {'network': {'id': 'a19261d8-bb8b-43ec-bce3-46fc3ba4a62b', 'name': 'ctest-dsnat_vn-67961940', 'tenant_id': '6cad61b2bda14cada50a1d7a8b122a90', 'project_id': '6cad61b2bda14cada50a1d7a8b122a90', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-08888997', 'ctest-dsnat_vn-67961940'], 'port_security_enabled': True, 'description': ''}} 2025-08-04 04:13:16,348 - DEBUG - Response for create_subnet : {'subnet': {'id': '37a09b20-94be-496d-b6f8-02dafae06fda', 'name': '', 'tenant_id': '6cad61b2bda14cada50a1d7a8b122a90', 'network_id': 'a19261d8-bb8b-43ec-bce3-46fc3ba4a62b', 'ip_version': 4, 'cidr': '11.152.120.0/26', 'allocation_pools': [{'start': '11.152.120.2', 'end': '11.152.120.62'}], 'gateway_ip': '11.152.120.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '11.152.120.2', 'tags': [], 'project_id': '6cad61b2bda14cada50a1d7a8b122a90'}} 2025-08-04 04:13:16,375 - INFO - Created VN ctest-dsnat_vn-67961940 2025-08-04 04:13:16,389 - DEBUG - VN ctest-dsnat_vn-67961940 UUID is a19261d8-bb8b-43ec-bce3-46fc3ba4a62b 2025-08-04 04:13:16,604 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/a19261d8-bb8b-43ec-bce3-46fc3ba4a62b 2025-08-04 04:13:16,932 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/a19261d8-bb8b-43ec-bce3-46fc3ba4a62b 2025-08-04 04:13:16,973 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/323e3605-4614-40f8-bc47-003518968558 2025-08-04 04:13:18,268 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3750-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3750-1) 2025-08-04 04:13:19,659 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3750-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3750-2) 2025-08-04 04:13:19,660 - INFO - Waiting for VM ctest-dsnat-vm1-55828838 to be up.. 2025-08-04 04:13:19,757 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-08-04 04:13:24,853 - DEBUG - VM is in ACTIVE state now 2025-08-04 04:13:24,853 - INFO - VM name : ctest-dsnat-vm1-55828838 2025-08-04 04:13:24,949 - DEBUG - VM ctest-dsnat-vm1-55828838 ID is 84ee303c-f70d-45ec-8175-615148a2d4b3 2025-08-04 04:13:24,982 - DEBUG - VM ctest-dsnat-vm1-55828838 launched on Node an-jenkins-deploy-platform-ansible-os-3750-1 2025-08-04 04:13:25,098 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine/84ee303c-f70d-45ec-8175-615148a2d4b3 2025-08-04 04:13:25,452 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine/84ee303c-f70d-45ec-8175-615148a2d4b3 2025-08-04 04:13:25,488 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine-interface/7be22b54-63e4-45c2-a6fe-15835aa8d2ef 2025-08-04 04:13:28,683 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-08-04 04:13:28,683 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:28,706 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:32,778 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-08-04 04:13:32,778 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:32,798 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:36,882 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-08-04 04:13:36,882 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:36,902 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:40,978 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1004ms') 2025-08-04 04:13:40,979 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:40,998 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:45,072 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms') 2025-08-04 04:13:45,073 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:45,095 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:49,192 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1033ms') 2025-08-04 04:13:49,192 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:49,212 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:53,297 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-08-04 04:13:53,297 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:53,315 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:13:57,395 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-08-04 04:13:57,395 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:13:57,414 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:14:01,488 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1005ms') 2025-08-04 04:14:01,488 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 failed! 2025-08-04 04:14:01,508 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-08888997:ctest-dsnat_vn-67961940 is 11.152.120.1 and allocation pool is NOT set 2025-08-04 04:14:03,573 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.27 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=4.88 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 4.881/5.574/6.267/0.693 ms') 2025-08-04 04:14:03,573 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-55828838 passed 2025-08-04 04:14:03,652 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 04:14:03,652 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-55828838, IP 11.152.120.3, Port 22 2025-08-04 04:14:03,720 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 04:14:03,816 - DEBUG - VM ctest-dsnat-vm1-55828838 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 04:14:08,817 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 04:14:08,818 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-55828838, IP 11.152.120.3, Port 22 2025-08-04 04:14:08,883 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 04:14:08,972 - DEBUG - VM ctest-dsnat-vm1-55828838 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 04:14:13,972 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 04:14:13,973 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-55828838, IP 11.152.120.3, Port 22 2025-08-04 04:14:14,081 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 04:14:14,170 - DEBUG - VM ctest-dsnat-vm1-55828838 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 04:14:19,170 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 04:14:19,171 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-55828838, IP 11.152.120.3, Port 22 2025-08-04 04:14:19,240 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-04 04:14:19,336 - DEBUG - VM ctest-dsnat-vm1-55828838 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-04 04:14:24,337 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 04:14:24,337 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-55828838, IP 11.152.120.3, Port 22 2025-08-04 04:14:24,508 - DEBUG - VM ctest-dsnat-vm1-55828838 is ready for SSH connections 2025-08-04 04:14:24,508 - INFO - Waiting for VM ctest-dsnat-vm2-91081625 to be up.. 2025-08-04 04:14:24,611 - DEBUG - VM is in ACTIVE state now 2025-08-04 04:14:24,612 - INFO - VM name : ctest-dsnat-vm2-91081625 2025-08-04 04:14:24,715 - DEBUG - VM ctest-dsnat-vm2-91081625 ID is 9fb37194-85bf-4c52-a041-35387ca1ef84 2025-08-04 04:14:24,715 - DEBUG - VM ctest-dsnat-vm2-91081625 launched on Node an-jenkins-deploy-platform-ansible-os-3750-2 2025-08-04 04:14:24,819 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine/9fb37194-85bf-4c52-a041-35387ca1ef84 2025-08-04 04:14:24,836 - DEBUG - Requesting: http://10.0.0.61:8082/virtual-machine-interface/115e585d-36f5-4943-ae97-179d78fabc53 2025-08-04 04:14:26,027 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=8.02 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.96 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1000ms\r\nrtt min/avg/max/mdev = 1.960/4.990/8.021/3.030 ms') 2025-08-04 04:14:26,027 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm2-91081625 passed 2025-08-04 04:14:26,120 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-04 04:14:26,120 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm2-91081625, IP 11.152.120.4, Port 22 2025-08-04 04:14:26,281 - DEBUG - VM ctest-dsnat-vm2-91081625 is ready for SSH connections 2025-08-04 04:14:26,391 - DEBUG - command output is cat: /tmp/iperf_server_ctest-random-36182052.pid: No such file or directory kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] 2025-08-04 04:14:26,509 - DEBUG - command output is 2025-08-04 04:14:26,509 - INFO - Starting Iperf3 on ctest-dsnat-vm1-55828838, args: --port 6201 --length 1048576 --time 10 2025-08-04 04:14:26,509 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result 2025-08-04 04:14:26,509 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'], on server 10.0.0.57 2025-08-04 04:14:26,614 - DEBUG - command output is [''] 2025-08-04 04:14:26,614 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 04:14:26,614 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result & echo $! > /tmp/iperf_client_ctest-random-36182052.pid 2025-08-04 04:14:44,260 - ERROR - Exception occured while running cmds ['iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'] 2025-08-04 04:14:44,261 - ERROR - Timeout opening channel. Traceback (most recent call last): File "/contrail-test/tcutils/fabutils.py", line 169, in remote_cmd output = _run(cmd, timeout=timeout, pty=not as_daemon, shell=shell) File "/usr/local/lib/python3.6/site-packages/fabric/network.py", line 692, in host_prompting_wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 1153, in sudo capture_buffer_size=capture_buffer_size, File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 935, in _run_command channel=default_channel(), command=wrapped_command, pty=pty, File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 435, in default_channel chan = _open_session() File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session session = transport.open_session(timeout=env.timeout) File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session timeout=timeout, File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1050, in open_channel raise e File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 2109, in run ptype, m = self.packetizer.read_message() File "/usr/local/lib/python3.6/site-packages/paramiko/packet.py", line 459, in read_message header = self.read_all(self.__block_size_in, check_rekey=True) File "/usr/local/lib/python3.6/site-packages/paramiko/packet.py", line 303, in read_all raise EOFError() EOFError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 435, in default_channel chan = _open_session() File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session session = transport.open_session(timeout=env.timeout) File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session timeout=timeout, File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1013, in open_channel raise SSHException("SSH session not active") paramiko.ssh_exception.SSHException: SSH session not active During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/contrail-test/fixtures/vm_test.py", line 2292, in run_cmd_on_vm logger=self.logger File "/contrail-test/tcutils/fabutils.py", line 178, in remote_cmd output = _run(cmd, timeout=timeout, pty=not as_daemon, shell=shell) File "/usr/local/lib/python3.6/site-packages/fabric/network.py", line 692, in host_prompting_wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 1153, in sudo capture_buffer_size=capture_buffer_size, File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 935, in _run_command channel=default_channel(), command=wrapped_command, pty=pty, File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 440, in default_channel chan = _open_session() File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session session = transport.open_session(timeout=env.timeout) File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session timeout=timeout, File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1054, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel. 2025-08-04 04:14:47,675 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples 2025-08-04 04:14:47,743 - DEBUG - command output is /bin/bash: line 1: kill: (60854) - No such process 2025-08-04 04:14:47,861 - DEBUG - command output is 2025-08-04 04:14:47,861 - INFO - Nat port being used for the flow is [] 2025-08-04 04:14:52,931 - DEBUG - command output is /bin/bash: line 1: kill: (60854) - No such process 2025-08-04 04:14:53,034 - DEBUG - command output is 2025-08-04 04:14:53,034 - INFO - Starting Iperf3 on ctest-dsnat-vm1-55828838, args: --port 6201 --length 1048576 --time 10 2025-08-04 04:14:53,034 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result 2025-08-04 04:14:53,034 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'], on server 10.0.0.57 2025-08-04 04:14:53,143 - DEBUG - command output is [''] 2025-08-04 04:14:53,144 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 04:14:53,144 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result & echo $! > /tmp/iperf_client_ctest-random-36182052.pid 2025-08-04 04:14:53,715 - DEBUG - None 2025-08-04 04:14:57,147 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples 2025-08-04 04:14:57,219 - DEBUG - command output is /bin/bash: line 1: kill: (61005) - No such process 2025-08-04 04:14:57,333 - DEBUG - command output is 2025-08-04 04:14:57,333 - INFO - Nat port being used for the flow is [] 2025-08-04 04:15:02,405 - DEBUG - command output is /bin/bash: line 1: kill: (61005) - No such process 2025-08-04 04:15:02,517 - DEBUG - command output is 2025-08-04 04:15:02,517 - INFO - Starting Iperf3 on ctest-dsnat-vm1-55828838, args: --port 6201 --length 1048576 --time 10 2025-08-04 04:15:02,517 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result 2025-08-04 04:15:02,517 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'], on server 10.0.0.57 2025-08-04 04:15:02,619 - DEBUG - command output is [''] 2025-08-04 04:15:02,619 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 04:15:02,619 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result & echo $! > /tmp/iperf_client_ctest-random-36182052.pid 2025-08-04 04:15:03,168 - DEBUG - None 2025-08-04 04:15:06,584 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples 2025-08-04 04:15:06,654 - DEBUG - command output is /bin/bash: line 1: kill: (61044) - No such process 2025-08-04 04:15:06,772 - DEBUG - command output is 2025-08-04 04:15:06,772 - INFO - Nat port being used for the flow is [] 2025-08-04 04:15:11,842 - DEBUG - command output is /bin/bash: line 1: kill: (61044) - No such process 2025-08-04 04:15:11,951 - DEBUG - command output is 2025-08-04 04:15:11,951 - INFO - Starting Iperf3 on ctest-dsnat-vm1-55828838, args: --port 6201 --length 1048576 --time 10 2025-08-04 04:15:11,951 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result 2025-08-04 04:15:11,952 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'], on server 10.0.0.57 2025-08-04 04:15:12,058 - DEBUG - command output is [''] 2025-08-04 04:15:12,058 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 04:15:12,058 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result & echo $! > /tmp/iperf_client_ctest-random-36182052.pid 2025-08-04 04:15:12,545 - DEBUG - None 2025-08-04 04:15:15,929 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples 2025-08-04 04:15:15,999 - DEBUG - command output is /bin/bash: line 1: kill: (61167) - No such process 2025-08-04 04:15:16,107 - DEBUG - command output is 2025-08-04 04:15:16,107 - INFO - Nat port being used for the flow is [] 2025-08-04 04:15:21,179 - DEBUG - command output is /bin/bash: line 1: kill: (61167) - No such process 2025-08-04 04:15:21,293 - DEBUG - command output is 2025-08-04 04:15:21,293 - INFO - Starting Iperf3 on ctest-dsnat-vm1-55828838, args: --port 6201 --length 1048576 --time 10 2025-08-04 04:15:21,293 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result 2025-08-04 04:15:21,294 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'], on server 10.0.0.57 2025-08-04 04:15:21,394 - DEBUG - command output is [''] 2025-08-04 04:15:21,395 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 04:15:21,395 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result & echo $! > /tmp/iperf_client_ctest-random-36182052.pid 2025-08-04 04:15:21,938 - DEBUG - None 2025-08-04 04:15:25,310 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples 2025-08-04 04:15:25,378 - DEBUG - command output is /bin/bash: line 1: kill: (61251) - No such process 2025-08-04 04:15:25,487 - DEBUG - command output is 2025-08-04 04:15:25,487 - INFO - Nat port being used for the flow is [] 2025-08-04 04:15:30,553 - DEBUG - command output is /bin/bash: line 1: kill: (61251) - No such process 2025-08-04 04:15:30,662 - DEBUG - command output is 2025-08-04 04:15:30,662 - INFO - Starting Iperf3 on ctest-dsnat-vm1-55828838, args: --port 6201 --length 1048576 --time 10 2025-08-04 04:15:30,662 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result 2025-08-04 04:15:30,662 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result'], on server 10.0.0.57 2025-08-04 04:15:30,770 - DEBUG - command output is [''] 2025-08-04 04:15:30,770 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123 2025-08-04 04:15:30,770 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-36182052.log 2>/tmp/iper3_ctest-random-36182052.result & echo $! > /tmp/iperf_client_ctest-random-36182052.pid 2025-08-04 04:15:31,290 - DEBUG - None 2025-08-04 04:15:34,655 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples 2025-08-04 04:15:34,723 - DEBUG - command output is /bin/bash: line 1: kill: (61302) - No such process 2025-08-04 04:15:34,838 - DEBUG - command output is 2025-08-04 04:15:34,838 - INFO - Nat port being used for the flow is [] 2025-08-04 04:15:34,942 - DEBUG - command output is /bin/bash: line 1: kill: (61302) - No such process 2025-08-04 04:15:35,056 - DEBUG - command output is 2025-08-04 04:15:35,056 - INFO - Deleting VM ctest-dsnat-vm2-91081625 2025-08-04 04:15:35,144 - INFO - Deleting VM ctest-dsnat-vm1-55828838 2025-08-04 04:15:35,220 - INFO - Deleting VN ctest-dsnat_vn-67961940 2025-08-04 04:15:35,272 - DEBUG - VN a19261d8-bb8b-43ec-bce3-46fc3ba4a62b still in use: Unable to complete operation on network a19261d8-bb8b-43ec-bce3-46fc3ba4a62b. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-7d03e38a-8b81-4588-ae3b-6b89c6d8badc'] 2025-08-04 04:15:35,272 - WARNING - Deleting VN ctest-dsnat_vn-67961940 failed..Will retry 2025-08-04 04:15:37,560 - DEBUG - Response for deleting network () 2025-08-04 04:15:38,848 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Mon Aug 4 04:15:34 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/dsnat/test_dsnat.py in test_dsnat_tcp_basic(self=) 108 self.iperf = None 109 110 assert self.verify_flow_with_port(vm1_fixture, vm2_fixture, port_range, **traffic) 111 112 @preposttest_wrapper self = self.verify_flow_with_port = .deco_retry.....test_dsnat_tcp_basic[sanity] id=0x7f887ff0e748>> vm1_fixture = vm2_fixture = port_range = [65000, 65001, 65002, 65003, 65004, 65005, 65006, 65007, 65008, 65009] traffic = {'port': 6201, 'udp': False} AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/dsnat/test_dsnat.py", line 110, in test_dsnat_tcp_basic assert self.verify_flow_with_port(vm1_fixture, vm2_fixture, port_range, **traffic) AssertionError 2025-08-04 04:15:38,848 - DEBUG - Skipping xmpp flap check 2025-08-04 04:15:38,848 - INFO -
2025-08-04 04:15:38,848 - INFO - END TEST : test_dsnat_tcp_basic : FAILED[0:02:25]
2025-08-04 04:15:38,848 - INFO - -------------------------------------------------------------------------------- 2025-08-04 04:15:39,682 - INFO - Deleted project: ctest-TestDSNAT-08888997, ID : 6cad61b2-bda1-4cad-a50a-1d7a8b122a90