2025-07-22 15:15:50,106 - INFO - Domain Default found not creating
2025-07-22 15:15:50,265 - INFO - Project ctest-TestDSNAT-56449480 not found, creating it
2025-07-22 15:15:50,951 - INFO - Created Project:ctest-TestDSNAT-56449480, ID : 5a73d6f7-d70f-41be-b7cf-4b29638c4962
2025-07-22 15:15:52,398 - INFO - ================================================================================
2025-07-22 15:15:52,398 - INFO - STARTING TEST : test_dsnat_basic
2025-07-22 15:15:52,398 - INFO - TEST DESCRIPTION :
create a VN and enable fabric SNAT
launch two VMs in that VN
verify ping between the VN and ping to the external IP
disable fabric SNAT
verify that the ping the external IP fails
2025-07-22 15:15:53,701 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with
2025-07-22 15:15:53,701 - INFO - Initial checks done. Running the testcase now
2025-07-22 15:15:53,701 - INFO -
2025-07-22 15:15:53,701 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance
2025-07-22 15:15:54,557 - DEBUG - Response for create_network : {'network': {'id': '787e7719-5b25-4dd3-8083-901eb4eac88d', 'name': 'ctest-dsnat_vn-59716798', 'tenant_id': '5a73d6f7d70f41beb7cf4b29638c4962', 'project_id': '5a73d6f7d70f41beb7cf4b29638c4962', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-56449480', 'ctest-dsnat_vn-59716798'], 'port_security_enabled': True, 'description': ''}}
2025-07-22 15:15:54,891 - DEBUG - Response for create_subnet : {'subnet': {'id': '1e20f3ff-223b-4572-bb56-7621db5e7eb2', 'name': '', 'tenant_id': '5a73d6f7d70f41beb7cf4b29638c4962', 'network_id': '787e7719-5b25-4dd3-8083-901eb4eac88d', 'ip_version': 4, 'cidr': '169.188.204.192/26', 'allocation_pools': [{'start': '169.188.204.194', 'end': '169.188.204.254'}], 'gateway_ip': '169.188.204.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '169.188.204.194', 'tags': [], 'project_id': '5a73d6f7d70f41beb7cf4b29638c4962'}}
2025-07-22 15:15:54,920 - INFO - Created VN ctest-dsnat_vn-59716798
2025-07-22 15:15:54,937 - DEBUG - VN ctest-dsnat_vn-59716798 UUID is 787e7719-5b25-4dd3-8083-901eb4eac88d
2025-07-22 15:15:55,158 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/787e7719-5b25-4dd3-8083-901eb4eac88d
2025-07-22 15:15:55,466 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/787e7719-5b25-4dd3-8083-901eb4eac88d
2025-07-22 15:15:55,504 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/a724a9db-a51e-4401-9819-6af4003cb425
2025-07-22 15:15:55,635 - DEBUG - Services list from nova: [, , , ]
2025-07-22 15:15:55,658 - DEBUG - Image by name ubuntu not found
2025-07-22 15:15:55,658 - DEBUG - Installing image ubuntu
2025-07-22 15:15:55,658 - DEBUG - Image info {'name': 'ubuntu.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'phusion-baseimage-enablesshd'}
2025-07-22 15:15:55,659 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/converts//ubuntu.vmdk.gz
2025-07-22 15:15:55,659 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/converts//ubuntu.vmdk.gz
2025-07-22 15:15:55,669 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/converts//ubuntu.vmdk.gz -O /contrail-test/images/ubuntu.vmdk.gz
2025-07-22 15:15:55,698 - DEBUG - Image has been downloaded to /contrail-test/images/ubuntu.vmdk.gz
2025-07-22 15:15:55,699 - DEBUG - Unzip image
2025-07-22 15:15:55,699 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu.vmdk.gz
2025-07-22 15:15:55,710 - DEBUG - Try to create image in user context
2025-07-22 15:15:55,786 - INFO - Deleting VN ctest-dsnat_vn-59716798
2025-07-22 15:15:56,006 - DEBUG - Response for deleting network ()
2025-07-22 15:15:57,269 - ERROR - Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/scripts/dsnat/test_dsnat.py", line 30, in test_dsnat_basic
image_name='ubuntu')
File "/contrail-test/common/base.py", line 281, in create_vm
**kwargs)
File "/contrail-test/common/base.py", line 254, in create_only_vm
vm_obj.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/ubuntu.vmdk'
2025-07-22 15:15:57,284 - DEBUG - No XMPP flaps were noticed during the test
2025-07-22 15:15:57,284 - INFO -
2025-07-22 15:15:57,284 - INFO - END TEST : test_dsnat_basic : FAILED[0:00:05]
2025-07-22 15:15:57,284 - INFO - --------------------------------------------------------------------------------
2025-07-22 15:15:57,288 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']}
2025-07-22 15:15:57,288 - INFO - ================================================================================
2025-07-22 15:15:57,288 - INFO - STARTING TEST : test_dsnat_bug_1749695
2025-07-22 15:15:57,288 - INFO - TEST DESCRIPTION :
Testcase to verify the bug 1749695
create test VN , associate a policy to allow all traffic between VN and ip fabric
associate the policy to the test VN and fabric VN
set the test VN fabric mode as l3
verify the ping succeeds for intra VN and to the fabric IP
2025-07-22 15:15:58,564 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with
2025-07-22 15:15:58,564 - INFO - Initial checks done. Running the testcase now
2025-07-22 15:15:58,564 - INFO -
2025-07-22 15:15:58,564 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance
2025-07-22 15:15:58,928 - DEBUG - Response for create_network : {'network': {'id': 'cd17e757-dbbd-4e96-83a8-905a19b8bb35', 'name': 'ctest-dsnat_vn-49253853', 'tenant_id': '5a73d6f7d70f41beb7cf4b29638c4962', 'project_id': '5a73d6f7d70f41beb7cf4b29638c4962', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-56449480', 'ctest-dsnat_vn-49253853'], 'port_security_enabled': True, 'description': ''}}
2025-07-22 15:15:59,219 - DEBUG - Response for create_subnet : {'subnet': {'id': '24bf4d76-7078-4a5c-9a01-64590fb712b5', 'name': '', 'tenant_id': '5a73d6f7d70f41beb7cf4b29638c4962', 'network_id': 'cd17e757-dbbd-4e96-83a8-905a19b8bb35', 'ip_version': 4, 'cidr': '54.34.36.0/26', 'allocation_pools': [{'start': '54.34.36.2', 'end': '54.34.36.62'}], 'gateway_ip': '54.34.36.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '54.34.36.2', 'tags': [], 'project_id': '5a73d6f7d70f41beb7cf4b29638c4962'}}
2025-07-22 15:15:59,243 - INFO - Created VN ctest-dsnat_vn-49253853
2025-07-22 15:15:59,258 - DEBUG - VN ctest-dsnat_vn-49253853 UUID is cd17e757-dbbd-4e96-83a8-905a19b8bb35
2025-07-22 15:15:59,490 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/cd17e757-dbbd-4e96-83a8-905a19b8bb35
2025-07-22 15:15:59,521 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/b5f137aa-ed95-4314-9a24-31a46733fc26
2025-07-22 15:15:59,749 - DEBUG - Image by name ubuntu-traffic not found
2025-07-22 15:15:59,749 - DEBUG - Installing image ubuntu-traffic
2025-07-22 15:15:59,749 - DEBUG - Image info {'name': 'ubuntu-traffic.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_small', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'ubuntu-traffic-docker', 'vmdk': 'ubuntu-traffic-disk1.vmdk', 'shrinked_vmdk': "'no'"}
2025-07-22 15:15:59,749 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/converts//ubuntu-traffic.vmdk.gz
2025-07-22 15:15:59,749 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/converts//ubuntu-traffic.vmdk.gz
2025-07-22 15:15:59,759 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/converts//ubuntu-traffic.vmdk.gz -O /contrail-test/images/ubuntu-traffic.vmdk.gz
2025-07-22 15:16:04,949 - DEBUG - Image has been downloaded to /contrail-test/images/ubuntu-traffic.vmdk.gz
2025-07-22 15:16:04,950 - DEBUG - Unzip image
2025-07-22 15:16:04,950 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu-traffic.vmdk.gz
2025-07-22 15:16:22,405 - DEBUG - Try to create image in user context
2025-07-22 15:16:32,912 - DEBUG - Image has been created in user context
2025-07-22 15:16:32,913 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: rm -f /contrail-test/images/ubuntu-traffic.vmdk
2025-07-22 15:16:33,138 - DEBUG - Downloaded image has been removed
2025-07-22 15:16:33,152 - DEBUG - Uploaded image status = active
2025-07-22 15:16:34,348 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1)
2025-07-22 15:16:36,033 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2)
2025-07-22 15:16:36,033 - INFO - Waiting for VM ctest-TestDSNAT-56449480-21080369 to be up..
2025-07-22 15:16:36,184 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:16:41,346 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:16:46,503 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:16:51,660 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:16:56,804 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:01,957 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:07,130 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:12,272 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:17,359 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:22,459 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:27,571 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:32,654 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:37,748 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:42,840 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:47,936 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:53,025 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:17:58,131 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 15:18:03,217 - DEBUG - VM is in ACTIVE state now
2025-07-22 15:18:03,217 - INFO - VM name : ctest-TestDSNAT-56449480-21080369
2025-07-22 15:18:03,326 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 ID is 86004ff6-2871-4f9d-9692-0ff226f9f7fa
2025-07-22 15:18:03,355 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 launched on Node an-jenkins-deploy-platform-ansible-os-3693-1
2025-07-22 15:18:03,447 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/86004ff6-2871-4f9d-9692-0ff226f9f7fa
2025-07-22 15:18:03,745 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/86004ff6-2871-4f9d-9692-0ff226f9f7fa
2025-07-22 15:18:03,783 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/59bb1187-6ea9-491a-aecf-5d317994aa1a
2025-07-22 15:18:07,006 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1001ms')
2025-07-22 15:18:07,006 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:07,025 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:11,110 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms')
2025-07-22 15:18:11,110 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:11,131 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:15,206 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-07-22 15:18:15,207 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:15,225 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:19,304 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms')
2025-07-22 15:18:19,304 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:19,324 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:23,398 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-07-22 15:18:23,398 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:23,425 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:27,527 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1037ms')
2025-07-22 15:18:27,527 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:27,549 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:31,622 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-07-22 15:18:31,622 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:31,643 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:35,718 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-07-22 15:18:35,718 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:35,736 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:39,807 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms')
2025-07-22 15:18:39,807 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 failed!
2025-07-22 15:18:39,827 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853 is 54.34.36.1 and allocation pool is NOT set
2025-07-22 15:18:41,892 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.10 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=6.75 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 6.095/6.420/6.745/0.325 ms')
2025-07-22 15:18:41,893 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-21080369 passed
2025-07-22 15:18:42,019 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:18:42,019 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-56449480-21080369, IP 54.34.36.3, Port 22
2025-07-22 15:18:42,088 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 15:18:42,204 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 15:18:47,205 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:18:47,205 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-56449480-21080369, IP 54.34.36.3, Port 22
2025-07-22 15:18:47,271 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 15:18:47,361 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 15:18:52,363 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:18:52,363 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-56449480-21080369, IP 54.34.36.3, Port 22
2025-07-22 15:18:52,431 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 15:18:52,521 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 15:18:57,522 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:18:57,523 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-56449480-21080369, IP 54.34.36.3, Port 22
2025-07-22 15:18:57,590 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 15:18:57,686 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 15:19:02,687 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:19:02,687 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-56449480-21080369, IP 54.34.36.3, Port 22
2025-07-22 15:19:02,855 - DEBUG - VM ctest-TestDSNAT-56449480-21080369 is ready for SSH connections
2025-07-22 15:19:02,855 - INFO - Waiting for VM ctest-TestDSNAT-56449480-51679815 to be up..
2025-07-22 15:19:02,944 - DEBUG - VM is in ACTIVE state now
2025-07-22 15:19:02,944 - INFO - VM name : ctest-TestDSNAT-56449480-51679815
2025-07-22 15:19:03,039 - DEBUG - VM ctest-TestDSNAT-56449480-51679815 ID is 81e63db2-9fb5-430c-81aa-de87adb2f924
2025-07-22 15:19:03,039 - DEBUG - VM ctest-TestDSNAT-56449480-51679815 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2
2025-07-22 15:19:03,126 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/81e63db2-9fb5-430c-81aa-de87adb2f924
2025-07-22 15:19:03,138 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/c2088f9e-ce90-4b80-bc64-472ed2d912aa
2025-07-22 15:19:04,359 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.64 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.26 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 1.261/3.451/5.642/2.190 ms')
2025-07-22 15:19:04,359 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestDSNAT-56449480-51679815 passed
2025-07-22 15:19:04,475 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:19:04,475 - DEBUG - Waiting to SSH to VM ctest-TestDSNAT-56449480-51679815, IP 54.34.36.4, Port 22
2025-07-22 15:19:04,637 - DEBUG - VM ctest-TestDSNAT-56449480-51679815 is ready for SSH connections
2025-07-22 15:19:04,793 - DEBUG - Fetched VN: default-domain:default-project:ip-fabric(7bd0b25e-be2b-42b9-83bc-fc1d418ae60e) with subnets None
2025-07-22 15:19:04,840 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestDSNAT-56449480:ctest-dsnat_vn-49253853, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:default-project:ip-fabric, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None]
2025-07-22 15:19:05,241 - INFO - Attaching policy ctest-test-dsnat-20728421 to vn ctest-dsnat_vn-49253853
2025-07-22 15:19:05,820 - INFO - Attaching policy ctest-test-dsnat-20728421 to vn ip-fabric
2025-07-22 15:19:06,116 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 15:19:06,117 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-07-22 15:19:29,108 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2017ms
2025-07-22 15:19:29,108 - WARNING - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-56449480-21080369 failed
2025-07-22 15:19:30,108 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 15:19:30,109 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-07-22 15:19:32,629 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
64 bytes from 10.20.0.17: icmp_req=1 ttl=62 time=4.35 ms
64 bytes from 10.20.0.17: icmp_req=2 ttl=62 time=0.578 ms
64 bytes from 10.20.0.17: icmp_req=3 ttl=62 time=0.784 ms
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 0.578/1.904/4.351/1.732 ms
2025-07-22 15:19:32,630 - INFO - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-56449480-21080369 passed
2025-07-22 15:19:32,748 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 15:19:32,748 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-07-22 15:19:36,275 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2015ms
2025-07-22 15:19:36,275 - WARNING - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-56449480-21080369 failed
2025-07-22 15:19:37,276 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 15:19:37,276 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-07-22 15:19:40,809 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms
2025-07-22 15:19:40,809 - WARNING - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-56449480-21080369 failed
2025-07-22 15:19:41,810 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 10.20.0.17, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 15:19:41,810 - DEBUG - ping -s 56 -c 3 -W 1 10.20.0.17
2025-07-22 15:19:44,322 - DEBUG - PING 10.20.0.17 (10.20.0.17) 56(84) bytes of data.
64 bytes from 10.20.0.17: icmp_req=1 ttl=62 time=1.81 ms
64 bytes from 10.20.0.17: icmp_req=2 ttl=62 time=0.595 ms
64 bytes from 10.20.0.17: icmp_req=3 ttl=62 time=0.565 ms
--- 10.20.0.17 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 0.565/0.992/1.818/0.584 ms
2025-07-22 15:19:44,322 - INFO - Ping to IP 10.20.0.17 from VM ctest-TestDSNAT-56449480-21080369 passed
2025-07-22 15:19:44,322 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 15:19:44,322 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 54.34.36.4, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 15:19:44,322 - DEBUG - ping -s 56 -c 3 -W 1 54.34.36.4
2025-07-22 15:19:46,870 - DEBUG - PING 54.34.36.4 (54.34.36.4) 56(84) bytes of data.
64 bytes from 54.34.36.4: icmp_req=1 ttl=63 time=13.0 ms
64 bytes from 54.34.36.4: icmp_req=2 ttl=63 time=1.29 ms
64 bytes from 54.34.36.4: icmp_req=3 ttl=63 time=1.26 ms
--- 54.34.36.4 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.269/5.221/13.099/5.570 ms
2025-07-22 15:19:46,870 - INFO - Ping to IP 54.34.36.4 from VM ctest-TestDSNAT-56449480-21080369 passed
2025-07-22 15:19:46,870 - INFO - Dettaching policy ctest-test-dsnat-20728421 from vn ip-fabric
2025-07-22 15:19:47,117 - INFO - Dettaching policy ctest-test-dsnat-20728421 from vn ctest-dsnat_vn-49253853
2025-07-22 15:19:47,389 - INFO - Deleted policy ctest-test-dsnat-20728421
2025-07-22 15:19:47,389 - INFO - Deleting VM ctest-TestDSNAT-56449480-51679815
2025-07-22 15:19:47,458 - INFO - Deleting VM ctest-TestDSNAT-56449480-21080369
2025-07-22 15:19:47,540 - INFO - Deleting VN ctest-dsnat_vn-49253853
2025-07-22 15:19:47,590 - DEBUG - VN cd17e757-dbbd-4e96-83a8-905a19b8bb35 still in use: Unable to complete operation on network cd17e757-dbbd-4e96-83a8-905a19b8bb35. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-051b4bfb-24cb-4abd-a3be-ff12ffc18d4b']
2025-07-22 15:19:47,590 - WARNING - Deleting VN ctest-dsnat_vn-49253853 failed..Will retry
2025-07-22 15:19:49,759 - DEBUG - Response for deleting network ()
2025-07-22 15:19:51,061 - DEBUG - No XMPP flaps were noticed during the test
2025-07-22 15:19:51,062 - INFO - END TEST : test_dsnat_bug_1749695 : PASSED[0:03:54]
2025-07-22 15:19:51,062 - INFO - --------------------------------------------------------------------------------
2025-07-22 15:19:51,838 - INFO - Deleted project: ctest-TestDSNAT-56449480, ID : 5a73d6f7-d70f-41be-b7cf-4b29638c4962
2025-07-22 17:51:41,126 - INFO - Domain Default found not creating
2025-07-22 17:51:41,292 - INFO - Project ctest-TestDSNAT-87744787 not found, creating it
2025-07-22 17:51:41,883 - INFO - Created Project:ctest-TestDSNAT-87744787, ID : 7996ea1d-fe74-4a2c-a231-926075380c2a
2025-07-22 17:51:43,434 - DEBUG - Services list from nova: [, , , ]
2025-07-22 17:51:43,434 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']}
2025-07-22 17:51:43,434 - INFO - ================================================================================
2025-07-22 17:51:43,434 - INFO - STARTING TEST : test_dsnat_tcp_basic
2025-07-22 17:51:43,435 - INFO - TEST DESCRIPTION :
Configure TCP port translation pool
send TCP traffic from the VM to the fabric IP
Verify the Port address translation happened and the
port being used in the range of configured
2025-07-22 17:51:44,681 - DEBUG - Skipping xmpp flap check
2025-07-22 17:51:44,681 - INFO - Initial checks done. Running the testcase now
2025-07-22 17:51:44,681 - INFO -
2025-07-22 17:51:45,148 - INFO - Create VN, enable FABRIC SNAT and verify its routing instance
2025-07-22 17:51:45,540 - DEBUG - Response for create_network : {'network': {'id': '4df11a57-ed11-4255-be76-e8f7e98f2da9', 'name': 'ctest-dsnat_vn-17347303', 'tenant_id': '7996ea1dfe744a2ca231926075380c2a', 'project_id': '7996ea1dfe744a2ca231926075380c2a', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDSNAT-87744787', 'ctest-dsnat_vn-17347303'], 'port_security_enabled': True, 'description': ''}}
2025-07-22 17:51:45,788 - DEBUG - Response for create_subnet : {'subnet': {'id': '4e43aaf1-099f-4c3b-b674-bd2ad3668e61', 'name': '', 'tenant_id': '7996ea1dfe744a2ca231926075380c2a', 'network_id': '4df11a57-ed11-4255-be76-e8f7e98f2da9', 'ip_version': 4, 'cidr': '166.243.124.128/26', 'allocation_pools': [{'start': '166.243.124.130', 'end': '166.243.124.190'}], 'gateway_ip': '166.243.124.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '166.243.124.130', 'tags': [], 'project_id': '7996ea1dfe744a2ca231926075380c2a'}}
2025-07-22 17:51:45,818 - INFO - Created VN ctest-dsnat_vn-17347303
2025-07-22 17:51:45,836 - DEBUG - VN ctest-dsnat_vn-17347303 UUID is 4df11a57-ed11-4255-be76-e8f7e98f2da9
2025-07-22 17:51:46,041 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/4df11a57-ed11-4255-be76-e8f7e98f2da9
2025-07-22 17:51:46,353 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/4df11a57-ed11-4255-be76-e8f7e98f2da9
2025-07-22 17:51:46,411 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/8249da89-87c7-4453-b021-d43bbf7e3189
2025-07-22 17:51:47,714 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1)
2025-07-22 17:51:49,106 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2)
2025-07-22 17:51:49,106 - INFO - Waiting for VM ctest-dsnat-vm1-88706784 to be up..
2025-07-22 17:51:49,212 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-07-22 17:51:54,303 - DEBUG - VM is in ACTIVE state now
2025-07-22 17:51:54,303 - INFO - VM name : ctest-dsnat-vm1-88706784
2025-07-22 17:51:54,384 - DEBUG - VM ctest-dsnat-vm1-88706784 ID is 4ae53a8a-1dee-47c0-b5a5-88a8174abeab
2025-07-22 17:51:54,408 - DEBUG - VM ctest-dsnat-vm1-88706784 launched on Node an-jenkins-deploy-platform-ansible-os-3693-1
2025-07-22 17:51:54,494 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/4ae53a8a-1dee-47c0-b5a5-88a8174abeab
2025-07-22 17:51:54,794 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/4ae53a8a-1dee-47c0-b5a5-88a8174abeab
2025-07-22 17:51:54,830 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/0a543a4e-dad5-46ae-8a2d-a8f9f205ea56
2025-07-22 17:51:58,015 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms')
2025-07-22 17:51:58,015 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:51:58,112 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:02,177 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms')
2025-07-22 17:52:02,177 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:02,195 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:06,274 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms')
2025-07-22 17:52:06,275 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:06,295 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:10,378 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms')
2025-07-22 17:52:10,378 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:10,396 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:14,466 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-07-22 17:52:14,466 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:14,484 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:18,569 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms')
2025-07-22 17:52:18,569 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:18,586 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:22,656 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-07-22 17:52:22,657 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:22,675 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:26,751 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms')
2025-07-22 17:52:26,751 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 failed!
2025-07-22 17:52:26,769 - DEBUG - Gateway for vn default-domain:ctest-TestDSNAT-87744787:ctest-dsnat_vn-17347303 is 166.243.124.129 and allocation pool is NOT set
2025-07-22 17:52:30,847 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=7.37 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1023ms\r\nrtt min/avg/max/mdev = 7.371/7.371/7.371/0.000 ms')
2025-07-22 17:52:30,847 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm1-88706784 passed
2025-07-22 17:52:30,923 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 17:52:30,925 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-88706784, IP 166.243.124.131, Port 22
2025-07-22 17:52:30,990 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 17:52:31,086 - DEBUG - VM ctest-dsnat-vm1-88706784 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 17:52:36,087 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 17:52:36,087 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-88706784, IP 166.243.124.131, Port 22
2025-07-22 17:52:36,165 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 17:52:36,254 - DEBUG - VM ctest-dsnat-vm1-88706784 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 17:52:41,255 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 17:52:41,255 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-88706784, IP 166.243.124.131, Port 22
2025-07-22 17:52:41,324 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 17:52:41,409 - DEBUG - VM ctest-dsnat-vm1-88706784 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 17:52:46,410 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 17:52:46,410 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-88706784, IP 166.243.124.131, Port 22
2025-07-22 17:52:46,479 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-07-22 17:52:46,563 - DEBUG - VM ctest-dsnat-vm1-88706784 is NOT ready for SSH connections, VM status: ACTIVE
2025-07-22 17:52:51,564 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 17:52:51,564 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm1-88706784, IP 166.243.124.131, Port 22
2025-07-22 17:52:51,731 - DEBUG - VM ctest-dsnat-vm1-88706784 is ready for SSH connections
2025-07-22 17:52:51,731 - INFO - Waiting for VM ctest-dsnat-vm2-82921712 to be up..
2025-07-22 17:52:51,811 - DEBUG - VM is in ACTIVE state now
2025-07-22 17:52:51,812 - INFO - VM name : ctest-dsnat-vm2-82921712
2025-07-22 17:52:51,888 - DEBUG - VM ctest-dsnat-vm2-82921712 ID is 3d684bc9-f6d5-40d0-82b5-201f547f0e4d
2025-07-22 17:52:51,888 - DEBUG - VM ctest-dsnat-vm2-82921712 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2
2025-07-22 17:52:51,963 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/3d684bc9-f6d5-40d0-82b5-201f547f0e4d
2025-07-22 17:52:51,976 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/8e440b92-d342-4e43-8f52-0d43c8ed1f9b
2025-07-22 17:52:53,158 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=8.64 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.76 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 2.762/5.702/8.642/2.940 ms')
2025-07-22 17:52:53,158 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-dsnat-vm2-82921712 passed
2025-07-22 17:52:53,230 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-07-22 17:52:53,230 - DEBUG - Waiting to SSH to VM ctest-dsnat-vm2-82921712, IP 166.243.124.132, Port 22
2025-07-22 17:52:53,404 - DEBUG - VM ctest-dsnat-vm2-82921712 is ready for SSH connections
2025-07-22 17:52:53,508 - DEBUG - command output is cat: /tmp/iperf_server_ctest-random-01539455.pid: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
2025-07-22 17:52:53,622 - DEBUG - command output is
2025-07-22 17:52:53,622 - INFO - Starting Iperf3 on ctest-dsnat-vm1-88706784, args: --port 6201 --length 1048576 --time 10
2025-07-22 17:52:53,622 - DEBUG - Iperf3 cmd : iperf3 -c 10.20.0.193 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-01539455.log 2>/tmp/iper3_ctest-random-01539455.result
2025-07-22 17:52:53,622 - INFO - Excecuting cmds, ['iperf3 -s --port 6201 1>/tmp/iper3_ctest-random-01539455.log 2>/tmp/iper3_ctest-random-01539455.result'], on server 10.0.0.27
2025-07-22 17:52:53,729 - DEBUG - command output is ['']
2025-07-22 17:52:53,729 - DEBUG - Running remote_cmd, Cmd : iperf3 -c 10.20.0.193 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-01539455.log 2>/tmp/iper3_ctest-random-01539455.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2025-07-22 17:52:53,729 - DEBUG - nohup iperf3 -c 10.20.0.193 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-01539455.log 2>/tmp/iper3_ctest-random-01539455.result & echo $! > /tmp/iperf_client_ctest-random-01539455.pid
2025-07-22 17:53:21,485 - ERROR - Exception occured while running cmds ['iperf3 -c 10.20.0.193 --port 6201 --length 1048576 --time 10 1>/tmp/iper3_ctest-random-01539455.log 2>/tmp/iper3_ctest-random-01539455.result']
2025-07-22 17:53:21,485 - ERROR - Timeout opening channel.
Traceback (most recent call last):
File "/contrail-test/tcutils/fabutils.py", line 169, in remote_cmd
output = _run(cmd, timeout=timeout, pty=not as_daemon, shell=shell)
File "/usr/local/lib/python3.6/site-packages/fabric/network.py", line 692, in host_prompting_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 1153, in sudo
capture_buffer_size=capture_buffer_size,
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 935, in _run_command
channel=default_channel(), command=wrapped_command, pty=pty,
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 435, in default_channel
chan = _open_session()
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session
session = transport.open_session(timeout=env.timeout)
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session
timeout=timeout,
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1050, in open_channel
raise e
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 2109, in run
ptype, m = self.packetizer.read_message()
File "/usr/local/lib/python3.6/site-packages/paramiko/packet.py", line 459, in read_message
header = self.read_all(self.__block_size_in, check_rekey=True)
File "/usr/local/lib/python3.6/site-packages/paramiko/packet.py", line 303, in read_all
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 435, in default_channel
chan = _open_session()
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session
session = transport.open_session(timeout=env.timeout)
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session
timeout=timeout,
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1013, in open_channel
raise SSHException("SSH session not active")
paramiko.ssh_exception.SSHException: SSH session not active
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/contrail-test/fixtures/vm_test.py", line 2287, in run_cmd_on_vm
logger=self.logger
File "/contrail-test/tcutils/fabutils.py", line 178, in remote_cmd
output = _run(cmd, timeout=timeout, pty=not as_daemon, shell=shell)
File "/usr/local/lib/python3.6/site-packages/fabric/network.py", line 692, in host_prompting_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 1153, in sudo
capture_buffer_size=capture_buffer_size,
File "/usr/local/lib/python3.6/site-packages/fabric/operations.py", line 935, in _run_command
channel=default_channel(), command=wrapped_command, pty=pty,
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 440, in default_channel
chan = _open_session()
File "/usr/local/lib/python3.6/site-packages/fabric/state.py", line 420, in _open_session
session = transport.open_session(timeout=env.timeout)
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 923, in open_session
timeout=timeout,
File "/usr/local/lib/python3.6/site-packages/paramiko/transport.py", line 1054, in open_channel
raise SSHException("Timeout opening channel.")
paramiko.ssh_exception.SSHException: Timeout opening channel.
2025-07-22 17:53:25,031 - DEBUG - command output is /bin/bash: line 1: kill: (72155) - No such process
2025-07-22 17:53:25,132 - DEBUG - command output is
2025-07-22 17:53:25,133 - INFO - Deleting VM ctest-dsnat-vm2-82921712
2025-07-22 17:53:25,245 - INFO - Deleting VM ctest-dsnat-vm1-88706784
2025-07-22 17:53:25,346 - INFO - Deleting VN ctest-dsnat_vn-17347303
2025-07-22 17:53:25,405 - DEBUG - VN 4df11a57-ed11-4255-be76-e8f7e98f2da9 still in use: Unable to complete operation on network 4df11a57-ed11-4255-be76-e8f7e98f2da9. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-f7573518-a1d7-4b9f-8609-b94bacebf042']
2025-07-22 17:53:25,406 - WARNING - Deleting VN ctest-dsnat_vn-17347303 failed..Will retry
2025-07-22 17:53:27,760 - DEBUG - Response for deleting network ()
2025-07-22 17:53:29,041 - ERROR - AttributeError
Python 3.6.8: /usr/bin/python3
Tue Jul 22 17:53:24 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/serial_scripts/dsnat/test_dsnat.py in test_dsnat_tcp_basic(self=)
108 self.iperf = None
109
110 assert self.verify_flow_with_port(vm1_fixture, vm2_fixture, port_range, **traffic)
111
112 @preposttest_wrapper
self =
self.verify_flow_with_port = .deco_retry.....test_dsnat_tcp_basic[sanity] id=0x7f536e3df908>>
vm1_fixture =
vm2_fixture =
port_range = [65000, 65001, 65002, 65003, 65004, 65005, 65006, 65007, 65008, 65009]
traffic = {'port': 6201, 'udp': False}
/contrail-test/tcutils/util.py in f_retry(*args=(, , , [65000, 65001, 65002, 65003, 65004, 65005, 65006, 65007, 65008, 65009]), **kwargs={'port': 6201, 'udp': False})
82 mtries, mdelay = tries, delay # make mutable
83
84 result = f(*args, **kwargs) # first attempt
85 rv = result
86 final = False
result undefined
f =
args = (, , , [65000, 65001, 65002, 65003, 65004, 65005, 65006, 65007, 65008, 65009])
kwargs = {'port': 6201, 'udp': False}
/contrail-test/common/dsnat/base.py in verify_flow_with_port(self=, client_vm_fix=, dst_vm_fix=, port_range=[65000, 65001, 65002, 65003, 65004, 65005, 65006, 65007, 65008, 65009], **traffic={'port': 6201, 'udp': False})
200 else:
201 proto = '6'
202 nat_port_used = self.get_nat_port_used_for_flow(client_vm_fix, proto, traffic['port'])
203 self.iperf.stop_iperf_on_server()
204 self.logger.info("Nat port being used for the flow is %s" %nat_port_used)
nat_port_used undefined
self =
self.get_nat_port_used_for_flow = >
client_vm_fix =
proto = '6'
traffic = {'port': 6201, 'udp': False}
/contrail-test/common/dsnat/base.py in get_nat_port_used_for_flow(self=, vm_fix=, proto='6', port=6201, container='agent')
185 self.connections, vm_fix.vm_node_ip))
186 flow_entry = server_fixture.get_flow_entry(dest_ip=vm_fix.vm_node_data_ip,
187 proto=proto, source_port = port,all_flows=True)
188 nat_port = []
189 for flow in flow_entry:
proto = '6'
source_port undefined
port = 6201
all_flows undefined
/contrail-test/fixtures/compute_node_test.py in get_flow_entry(self=, flow_table=, index=None, source_ip=None, dest_ip='10.20.0.25', source_port=6201, dest_port=None, proto='6', vrf_id=None, refresh=True, show_evicted=True, all_flows=True)
561 all_flow_list = []
562 for flow_entry_item in flow_table.items:
563 if reqd_entries.viewitems() <= flow_entry_item.viewitems():
564 forward_flow = FlowEntry(flow_entry_item)
565 if flow_entry_item['rflow'] != '-1':
reqd_entries = {'dip': '10.20.0.25', 'proto': '6', 'sport': '6201'}
reqd_entries.viewitems undefined
flow_entry_item = {'action': 'DROP', 'bytes': '54', 'd_vrf_id': '0', 'dip': '155.169.32.66', 'dport': '54036', 'drop_reason': 'Unknown', 'flags': ' ACTIVE | EVICTED ', 'gen_id': '1', 'index': '32', 'insight': '0', ...}
flow_entry_item.viewitems undefined
AttributeError: 'dict' object has no attribute 'viewitems'
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = 'Attribute not found.'
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
args = ("'dict' object has no attribute 'viewitems'",)
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/dsnat/test_dsnat.py", line 110, in test_dsnat_tcp_basic
assert self.verify_flow_with_port(vm1_fixture, vm2_fixture, port_range, **traffic)
File "/contrail-test/tcutils/util.py", line 84, in f_retry
result = f(*args, **kwargs) # first attempt
File "/contrail-test/common/dsnat/base.py", line 202, in verify_flow_with_port
nat_port_used = self.get_nat_port_used_for_flow(client_vm_fix, proto, traffic['port'])
File "/contrail-test/common/dsnat/base.py", line 187, in get_nat_port_used_for_flow
proto=proto, source_port = port,all_flows=True)
File "/contrail-test/fixtures/compute_node_test.py", line 563, in get_flow_entry
if reqd_entries.viewitems() <= flow_entry_item.viewitems():
AttributeError: 'dict' object has no attribute 'viewitems'
2025-07-22 17:53:29,042 - DEBUG - Skipping xmpp flap check
2025-07-22 17:53:29,042 - INFO -
2025-07-22 17:53:29,042 - INFO - END TEST : test_dsnat_tcp_basic : FAILED[0:01:46]
2025-07-22 17:53:29,043 - INFO - --------------------------------------------------------------------------------
2025-07-22 17:53:29,976 - INFO - Deleted project: ctest-TestDSNAT-87744787, ID : 7996ea1d-fe74-4a2c-a231-926075380c2a