is in ACTIVE state now
2025-08-07 02:29:53,556 - INFO - VM name : ctest-TestDSNAT-67738739-34054312
2025-08-07 02:29:53,651 - DEBUG - VM ctest-TestDSNAT-67738739-34054312 ID is a1f08848-1717-4137-a01e-89004efbe423
2025-08-07 02:29:53,651 - DEBUG - VM ctest-TestDSNAT-67738739-34054312 launched on Node an-jenkins-deploy-platform-ansible-os-3767-2
2025-08-07 02:29:53,739 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine/a1f08848-1717-4137-a01e-89004efbe423
2025-08-07 02:29:53,751 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine-interface/1747ce56-beb0-4a16-9008-b5741cc8d9ec
2025-08-07 02:29:54,961 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=10.6 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.43 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 1.432/6.001/10.570/4.569 ms')
2025-08-07 02:29:54,961 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-67738739-34054312 passed
2025-08-07 02:29:55,083 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 02:29:55,083 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-67738739-34054312, IP 134.210.24.4, Port 22
2025-08-07 02:29:55,255 - DEBUG - VM ctest-TestDSNAT-67738739-34054312 is ready for SSH connections
2025-08-07 02:29:55,410 - DEBUG - Fetched VN: default-domain:default-project:ip-fabric(efc777f4-6ce2-435f-9ae2-6a3dd4640831) with subnets None
2025-08-07 02:29:55,462 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestDSNAT-67738739:ctest-dsnat_vn-59190444, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:default-project:ip-fabric, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None]
2025-08-07 02:29:55,828 - INFO - Attaching policy ctest-test-dsnat-10551141 to vn ctest-dsnat_vn-59190444
2025-08-07 02:29:56,378 - INFO - Attaching policy ctest-test-dsnat-10551141 to vn ip-fabric
2025-08-07 02:29:56,699 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 02:29:56,699 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-08-07 02:30:20,990 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
64 bytes from 10.20.0.17: icmp_req=1 ttl=62 time=4.65 ms
64 bytes from 10.20.0.17: icmp_req=2 ttl=62 time=1.15 ms
64 bytes from 10.20.0.17: icmp_req=3 ttl=62 time=0.934 ms
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 0.934/2.245/4.653/1.705 ms
2025-08-07 02:30:20,990 - INFO - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-67738739-01819120 passed
2025-08-07 02:30:21,122 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 02:30:21,122 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-08-07 02:30:24,232 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
64 bytes from 10.20.0.17: icmp_req=1 ttl=62 time=3.19 ms
64 bytes from 10.20.0.17: icmp_req=2 ttl=62 time=1.12 ms
64 bytes from 10.20.0.17: icmp_req=3 ttl=62 time=0.709 ms
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2005ms
rtt min/avg/max/mdev = 0.709/1.677/3.199/1.090 ms
2025-08-07 02:30:24,232 - INFO - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-67738739-01819120 passed
2025-08-07 02:30:24,232 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 02:30:24,232 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 134.210.24.4, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 02:30:24,232 - DEBUG - ping -s 56 -c 3 -W 1 134.210.24.4
2025-08-07 02:30:26,798 - DEBUG - PING 134.210.24.4 (134.210.24.4) 56(84) bytes of data.
64 bytes from 134.210.24.4: icmp_req=1 ttl=63 time=7.80 ms
64 bytes from 134.210.24.4: icmp_req=2 ttl=63 time=1.31 ms
64 bytes from 134.210.24.4: icmp_req=3 ttl=63 time=1.32 ms
--- 134.210.24.4 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.312/3.478/7.801/3.056 ms
2025-08-07 02:30:26,798 - INFO - Ping to IP 134.210.24.4 from VM ctest-TestDSNAT-67738739-01819120 passed
2025-08-07 02:30:26,799 - INFO - Dettaching policy ctest-test-dsnat-10551141 from vn ip-fabric
2025-08-07 02:30:27,039 - INFO - Dettaching policy ctest-test-dsnat-10551141 from vn ctest-dsnat_vn-59190444
2025-08-07 02:30:27,343 - INFO - Deleted policy ctest-test-dsnat-10551141
2025-08-07 02:30:27,343 - INFO - Deleting VM ctest-TestDSNAT-67738739-34054312
2025-08-07 02:30:27,434 - INFO - Deleting VM ctest-TestDSNAT-67738739-01819120
2025-08-07 02:30:27,514 - INFO - Deleting VN ctest-dsnat_vn-59190444
2025-08-07 02:30:27,579 - DEBUG - VN 3a0b6aa5-a8b6-4d77-b5df-cb45c62c3293 still in use: Unable to complete operation on network 3a0b6aa5-a8b6-4d77-b5df-cb45c62c3293. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-fcbfd809-43ca-494c-9933-b7cd365b8c19']
2025-08-07 02:30:27,579 - WARNING - Deleting VN ctest-dsnat_vn-59190444 failed..Will retry
2025-08-07 02:30:29,856 - DEBUG - Response for deleting network ()
2025-08-07 02:30:31,144 - DEBUG - No XMPP flaps were noticed during the test
2025-08-07 02:30:31,144 - INFO - END TEST : test_dsnat_bug_1749695 : PASSED[0:03:30]
2025-08-07 02:30:31,144 - INFO - --------------------------------------------------------------------------------
2025-08-07 02:30:31,947 - INFO - Deleted project: ctest-TestDSNAT-67738739, ID : 4bbc631d-2332-4cef-aa12-2fd4be043d7d
2025-08-07 04:10:32,248 - INFO - Domain Default found not creating
2025-08-07 04:10:32,444 - INFO - Project ctest-TestDSNAT-91191756 not found, creating it
2025-08-07 04:10:33,062 - INFO - Created Project:ctest-TestDSNAT-91191756, ID : 7b18fc48-bc25-43da-b694-65a101e70853
2025-08-07 04:10:34,701 - DEBUG - Services list from nova: [, , , ]
2025-08-07 04:10:34,702 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3767-2', 'an-jenkins-deploy-platform-ansible-os-3767-1']}
2025-08-07 04:10:34,702 - INFO - ================================================================================
2025-08-07 04:10:34,702 - INFO - STARTING TEST : test_dsnat_tcp_basic
2025-08-07 04:10:34,703 - INFO - TEST DESCRIPTION :
Configure TCP port translation pool
send TCP traffic from the VM to the fabric IP
Verify the Port address translation happened and the
port being used in the range of configured
2025-08-07 04:10:35,985 - DEBUG - Skipping xmpp flap check
2025-08-07 04:10:35,985 - INFO - Initial checks done. Running the testcase now
2025-08-07 04:10:35,985 - INFO -
2025-08-07 04:10:36,498 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance
2025-08-07 04:10:37,006 - DEBUG - Response for create_network : {'network': {'id': '6c5934ab-6569-43a7-bd81-07b550fc474e', 'name': 'ctest-dsnat_vn-51289871', 'tenant_id': '7b18fc48bc2543dab69465a101e70853', 'project_id': '7b18fc48bc2543dab69465a101e70853', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-91191756', 'ctest-dsnat_vn-51289871'], 'port_security_enabled': True, 'description': ''}}
2025-08-07 04:10:37,353 - DEBUG - Response for create_subnet : {'subnet': {'id': 'f733c591-e836-4e55-bf57-d30c30564e5a', 'name': '', 'tenant_id': '7b18fc48bc2543dab69465a101e70853', 'network_id': '6c5934ab-6569-43a7-bd81-07b550fc474e', 'ip_version': 4, 'cidr': '81.131.128.192/26', 'allocation_pools': [{'start': '81.131.128.194', 'end': '81.131.128.254'}], 'gateway_ip': '81.131.128.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '81.131.128.194', 'tags': [], 'project_id': '7b18fc48bc2543dab69465a101e70853'}}
2025-08-07 04:10:37,380 - INFO - Created VN ctest-dsnat_vn-51289871
2025-08-07 04:10:37,396 - DEBUG - VN ctest-dsnat_vn-51289871 UUID is 6c5934ab-6569-43a7-bd81-07b550fc474e
2025-08-07 04:10:37,626 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/6c5934ab-6569-43a7-bd81-07b550fc474e
2025-08-07 04:10:38,002 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/6c5934ab-6569-43a7-bd81-07b550fc474e
2025-08-07 04:10:38,049 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/bdb9b77d-cb92-42a9-90c0-8b81c6bb5795
2025-08-07 04:10:39,407 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3767-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3767-1)
2025-08-07 04:10:40,974 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3767-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3767-2)
2025-08-07 04:10:40,974 - INFO - Waiting for VM ctest-dsnat-vm1-91581087 to be up..
2025-08-07 04:10:41,098 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-08-07 04:10:46,214 - DEBUG - VM is in ACTIVE state now
2025-08-07 04:10:46,214 - INFO - VM name : ctest-dsnat-vm1-91581087
2025-08-07 04:10:46,310 - DEBUG - VM ctest-dsnat-vm1-91581087 ID is 7772e9a8-39c6-46c5-b813-055695a33943
2025-08-07 04:10:46,337 - DEBUG - VM ctest-dsnat-vm1-91581087 launched on Node an-jenkins-deploy-platform-ansible-os-3767-1
2025-08-07 04:10:46,443 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine/7772e9a8-39c6-46c5-b813-055695a33943
2025-08-07 04:10:46,753 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine/7772e9a8-39c6-46c5-b813-055695a33943
2025-08-07 04:10:46,796 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine-interface/62153d34-0357-4c26-bf26-b0e3547afc4d
2025-08-07 04:10:50,014 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1028ms')
2025-08-07 04:10:50,014 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:10:50,035 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:10:54,149 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1029ms')
2025-08-07 04:10:54,149 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:10:54,173 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:10:58,274 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms')
2025-08-07 04:10:58,274 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:10:58,294 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:02,371 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms')
2025-08-07 04:11:02,372 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:02,422 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:06,498 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms')
2025-08-07 04:11:06,498 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:06,518 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:10,595 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms')
2025-08-07 04:11:10,596 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:10,612 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:14,692 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms')
2025-08-07 04:11:14,693 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:14,712 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:18,823 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms')
2025-08-07 04:11:18,823 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:18,844 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:22,942 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1027ms')
2025-08-07 04:11:22,942 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:22,964 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:27,038 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms')
2025-08-07 04:11:27,038 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:27,058 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:31,138 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms')
2025-08-07 04:11:31,138 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 failed!
2025-08-07 04:11:31,158 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-91191756:ctest-dsnat_vn-51289871 is 81.131.128.193 and allocation pool is NOT set
2025-08-07 04:11:33,233 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=9.43 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=4.34 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 4.344/6.886/9.429/2.542 ms')
2025-08-07 04:11:33,233 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-91581087 passed
2025-08-07 04:11:33,315 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:11:33,315 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-91581087, IP 81.131.128.195, Port 22
2025-08-07 04:11:33,387 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-08-07 04:11:33,494 - DEBUG - VM ctest-dsnat-vm1-91581087 is NOT ready for SSH connections, VM status: ACTIVE
2025-08-07 04:11:38,494 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:11:38,495 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-91581087, IP 81.131.128.195, Port 22
2025-08-07 04:11:38,565 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-08-07 04:11:38,659 - DEBUG - VM ctest-dsnat-vm1-91581087 is NOT ready for SSH connections, VM status: ACTIVE
2025-08-07 04:11:43,660 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:11:43,661 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-91581087, IP 81.131.128.195, Port 22
2025-08-07 04:11:43,731 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-08-07 04:11:43,838 - DEBUG - VM ctest-dsnat-vm1-91581087 is NOT ready for SSH connections, VM status: ACTIVE
2025-08-07 04:11:48,839 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:11:48,839 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-91581087, IP 81.131.128.195, Port 22
2025-08-07 04:11:48,909 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-08-07 04:11:49,006 - DEBUG - VM ctest-dsnat-vm1-91581087 is NOT ready for SSH connections, VM status: ACTIVE
2025-08-07 04:11:54,007 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:11:54,007 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-91581087, IP 81.131.128.195, Port 22
2025-08-07 04:11:54,078 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-08-07 04:11:54,170 - DEBUG - VM ctest-dsnat-vm1-91581087 is NOT ready for SSH connections, VM status: ACTIVE
2025-08-07 04:11:59,171 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:11:59,171 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-91581087, IP 81.131.128.195, Port 22
2025-08-07 04:11:59,372 - DEBUG - VM ctest-dsnat-vm1-91581087 is ready for SSH connections
2025-08-07 04:11:59,372 - INFO - Waiting for VM ctest-dsnat-vm2-60086708 to be up..
2025-08-07 04:11:59,474 - DEBUG - VM is in ACTIVE state now
2025-08-07 04:11:59,474 - INFO - VM name : ctest-dsnat-vm2-60086708
2025-08-07 04:11:59,570 - DEBUG - VM ctest-dsnat-vm2-60086708 ID is 340777bf-5927-465d-a12c-8ca91af3dfac
2025-08-07 04:11:59,570 - DEBUG - VM ctest-dsnat-vm2-60086708 launched on Node an-jenkins-deploy-platform-ansible-os-3767-2
2025-08-07 04:11:59,666 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine/340777bf-5927-465d-a12c-8ca91af3dfac
2025-08-07 04:11:59,680 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-machine-interface/58c11c85-7918-4cdf-a267-38c419ce27ec
2025-08-07 04:12:00,867 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=4.80 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.954 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.954/2.875/4.797/1.921 ms')
2025-08-07 04:12:00,867 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm2-60086708 passed
2025-08-07 04:12:00,958 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-07 04:12:00,958 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm2-60086708, IP 81.131.128.196, Port 22
2025-08-07 04:12:01,132 - DEBUG - VM ctest-dsnat-vm2-60086708 is ready for SSH connections
2025-08-07 04:12:01,241 - DEBUG - command output is cat: /tmp/iperf_server_ctest-random-07651854.pid: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
2025-08-07 04:12:01,355 - DEBUG - command output is
2025-08-07 04:12:01,355 - INFO - Starting Iperf3 on ctest-dsnat-vm1-91581087, args: --port 6201 --length 1048576 --time 10
2025-08-07 04:12:01,355 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result
2025-08-07 04:12:01,356 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result'], on server 10.0.0.61
2025-08-07 04:12:01,458 - DEBUG - command output is ['']
2025-08-07 04:12:01,458 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 04:12:01,458 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result & echo $! > /tmp/iperf_client_ctest-random-07651854.pid
2025-08-07 04:12:19,705 - ERROR - Exception occured while running cmds ['iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result']
2025-08-07 04:12:19,705 - ERROR - Timeout opening channel.
Traceback (most recent call last):
File "/contrail-test/tcutils/fabutils.py", line 169, in remote_cmd
output = _run(cmd, timeout=timeout, pty=not as_daemon, shell=shell)
File "/usr/local/lib/python3.6/site-packages/fabric/network.py", line 692, in host_prompting_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 1153, in sudo
capture_buffer_size=capture_buffer_size,
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 935, in _run_command
channel=default_channel(), command=wrapped_command, pty=pty,
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 435, in default_channel
chan = _open_session()
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session
session = transport.open_session(timeout=env.timeout)
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session
timeout=timeout,
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1050, in open_channel
raise e
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 2109, in run
ptype, m = self.packetizer.read_message()
File "/usr/local/lib/python3.6/site-packages/paramiko/packet.py", line 459, in read_message
header = self.read_all(self.__block_size_in, check_rekey=True)
File "/usr/local/lib/python3.6/site-packages/paramiko/packet.py", line 303, in read_all
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 435, in default_channel
chan = _open_session()
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session
session = transport.open_session(timeout=env.timeout)
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session
timeout=timeout,
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1013, in open_channel
raise SSHException("SSH session not active")
paramiko.ssh_exception.SSHException: SSH session not active
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/contrail-test/fixtures/vm_test.py", line 2292, in run_cmd_on_vm
logger=self.logger
File "/contrail-test/tcutils/fabutils.py", line 178, in remote_cmd
output = _run(cmd, timeout=timeout, pty=not as_daemon, shell=shell)
File "/usr/local/lib/python3.6/site-packages/fabric/network.py", line 692, in host_prompting_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 1153, in sudo
capture_buffer_size=capture_buffer_size,
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 935, in _run_command
channel=default_channel(), command=wrapped_command, pty=pty,
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 440, in default_channel
chan = _open_session()
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session
session = transport.open_session(timeout=env.timeout)
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session
timeout=timeout,
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1054, in open_channel
raise SSHException("Timeout opening channel.")
paramiko.ssh_exception.SSHException: Timeout opening channel.
2025-08-07 04:12:23,213 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples
2025-08-07 04:12:23,284 - DEBUG - command output is /bin/bash: line 1: kill: (63389) - No such process
2025-08-07 04:12:23,388 - DEBUG - command output is
2025-08-07 04:12:23,388 - INFO - Nat port being used for the flow is []
2025-08-07 04:12:28,456 - DEBUG - command output is /bin/bash: line 1: kill: (63389) - No such process
2025-08-07 04:12:28,570 - DEBUG - command output is
2025-08-07 04:12:28,570 - INFO - Starting Iperf3 on ctest-dsnat-vm1-91581087, args: --port 6201 --length 1048576 --time 10
2025-08-07 04:12:28,570 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result
2025-08-07 04:12:28,571 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result'], on server 10.0.0.61
2025-08-07 04:12:28,676 - DEBUG - command output is ['']
2025-08-07 04:12:28,677 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 04:12:28,677 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result & echo $! > /tmp/iperf_client_ctest-random-07651854.pid
2025-08-07 04:12:30,014 - DEBUG - None
2025-08-07 04:12:33,523 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples
2025-08-07 04:12:33,596 - DEBUG - command output is /bin/bash: line 1: kill: (63556) - No such process
2025-08-07 04:12:33,700 - DEBUG - command output is
2025-08-07 04:12:33,700 - INFO - Nat port being used for the flow is []
2025-08-07 04:12:38,772 - DEBUG - command output is /bin/bash: line 1: kill: (63556) - No such process
2025-08-07 04:12:38,887 - DEBUG - command output is
2025-08-07 04:12:38,887 - INFO - Starting Iperf3 on ctest-dsnat-vm1-91581087, args: --port 6201 --length 1048576 --time 10
2025-08-07 04:12:38,887 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result
2025-08-07 04:12:38,887 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result'], on server 10.0.0.61
2025-08-07 04:12:38,988 - DEBUG - command output is ['']
2025-08-07 04:12:38,988 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 04:12:38,988 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result & echo $! > /tmp/iperf_client_ctest-random-07651854.pid
2025-08-07 04:12:39,514 - DEBUG - None
2025-08-07 04:12:42,914 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples
2025-08-07 04:12:42,983 - DEBUG - command output is /bin/bash: line 1: kill: (63620) - No such process
2025-08-07 04:12:43,098 - DEBUG - command output is
2025-08-07 04:12:43,098 - INFO - Nat port being used for the flow is []
2025-08-07 04:12:48,167 - DEBUG - command output is /bin/bash: line 1: kill: (63620) - No such process
2025-08-07 04:12:48,271 - DEBUG - command output is
2025-08-07 04:12:48,271 - INFO - Starting Iperf3 on ctest-dsnat-vm1-91581087, args: --port 6201 --length 1048576 --time 10
2025-08-07 04:12:48,272 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result
2025-08-07 04:12:48,272 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result'], on server 10.0.0.61
2025-08-07 04:12:48,380 - DEBUG - command output is ['']
2025-08-07 04:12:48,380 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 04:12:48,380 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result & echo $! > /tmp/iperf_client_ctest-random-07651854.pid
2025-08-07 04:12:48,903 - DEBUG - None
2025-08-07 04:12:52,373 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples
2025-08-07 04:12:52,458 - DEBUG - command output is /bin/bash: line 1: kill: (63690) - No such process
2025-08-07 04:12:52,570 - DEBUG - command output is
2025-08-07 04:12:52,570 - INFO - Nat port being used for the flow is []
2025-08-07 04:12:57,640 - DEBUG - command output is /bin/bash: line 1: kill: (63690) - No such process
2025-08-07 04:12:57,753 - DEBUG - command output is
2025-08-07 04:12:57,754 - INFO - Starting Iperf3 on ctest-dsnat-vm1-91581087, args: --port 6201 --length 1048576 --time 10
2025-08-07 04:12:57,754 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result
2025-08-07 04:12:57,754 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result'], on server 10.0.0.61
2025-08-07 04:12:57,856 - DEBUG - command output is ['']
2025-08-07 04:12:57,856 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 04:12:57,856 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result & echo $! > /tmp/iperf_client_ctest-random-07651854.pid
2025-08-07 04:12:58,394 - DEBUG - None
2025-08-07 04:13:01,836 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples
2025-08-07 04:13:01,904 - DEBUG - command output is /bin/bash: line 1: kill: (63735) - No such process
2025-08-07 04:13:02,017 - DEBUG - command output is
2025-08-07 04:13:02,018 - INFO - Nat port being used for the flow is []
2025-08-07 04:13:07,098 - DEBUG - command output is /bin/bash: line 1: kill: (63735) - No such process
2025-08-07 04:13:07,211 - DEBUG - command output is
2025-08-07 04:13:07,211 - INFO - Starting Iperf3 on ctest-dsnat-vm1-91581087, args: --port 6201 --length 1048576 --time 10
2025-08-07 04:13:07,211 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result
2025-08-07 04:13:07,211 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result'], on server 10.0.0.61
2025-08-07 04:13:07,316 - DEBUG - command output is ['']
2025-08-07 04:13:07,316 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-07 04:13:07,316 - DEBUG - nohup iperf3 -c 10.20.0.201 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-07651854.log 2>/tmp/iper3_ctest-random-07651854.result & echo $! > /tmp/iperf_client_ctest-random-07651854.pid
2025-08-07 04:13:07,906 - DEBUG - None
2025-08-07 04:13:11,314 - DEBUG - Returns multiple flows, [], matching flow data as list of tuples
2025-08-07 04:13:11,384 - DEBUG - command output is /bin/bash: line 1: kill: (63828) - No such process
2025-08-07 04:13:11,497 - DEBUG - command output is
2025-08-07 04:13:11,498 - INFO - Nat port being used for the flow is []
2025-08-07 04:13:11,600 - DEBUG - command output is /bin/bash: line 1: kill: (63828) - No such process
2025-08-07 04:13:11,704 - DEBUG - command output is
2025-08-07 04:13:11,704 - INFO - Deleting VM ctest-dsnat-vm2-60086708
2025-08-07 04:13:11,797 - INFO - Deleting VM ctest-dsnat-vm1-91581087
2025-08-07 04:13:11,880 - INFO - Deleting VN ctest-dsnat_vn-51289871
2025-08-07 04:13:11,933 - DEBUG - VN 6c5934ab-6569-43a7-bd81-07b550fc474e still in use: Unable to complete operation on network 6c5934ab-6569-43a7-bd81-07b550fc474e. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-bda595db-d887-4998-aa2f-e706604d629f']
2025-08-07 04:13:11,933 - WARNING - Deleting VN ctest-dsnat_vn-51289871 failed..Will retry
2025-08-07 04:13:14,155 - DEBUG - Response for deleting network ()
2025-08-07 04:13:15,462 - ERROR - AssertionError
Python 3.6.8: /usr/bin/python3
Thu Aug 7 04:13:11 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/serial_scripts/dsnat/test_dsnat.py in test_dsnat_tcp_basic(self=)
108 self.iperf = None
109
110 assert self.verify_flow_with_port(vm1_fixture, vm2_fixture, port_range, **traffic)
111
112 @preposttest_wrapper
self =
self.verify_flow_with_port = .deco_retry.....test_dsnat_tcp_basic[sanity] id=0x7f0bc488b748>>
vm1_fixture =
vm2_fixture =
port_range = [65000, 65001, 65002, 65003, 65004, 65005, 65006, 65007, 65008, 65009]
traffic = {'port': 6201, 'udp': False}
AssertionError:
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = 'Assertion failed.'
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
args = ()
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/dsnat/test_dsnat.py", line 110, in test_dsnat_tcp_basic
assert self.verify_flow_with_port(vm1_fixture, vm2_fixture, port_range, **traffic)
AssertionError
2025-08-07 04:13:15,463 - DEBUG - Skipping xmpp flap check
2025-08-07 04:13:15,463 - INFO -
2025-08-07 04:13:15,463 - INFO - END TEST : test_dsnat_tcp_basic : FAILED[0:02:41]
2025-08-07 04:13:15,463 - INFO - --------------------------------------------------------------------------------
2025-08-07 04:13:16,277 - INFO - Deleted project: ctest-TestDSNAT-91191756, ID : 7b18fc48-bc25-43da-b694-65a101e70853