2025-09-11 03:19:04,377 - INFO - Domain Default found not creating
2025-09-11 03:19:04,525 - INFO - Project ctest-TestBGPaasZone-96156374 not found, creating it
2025-09-11 03:19:05,217 - INFO - Created Project:ctest-TestBGPaasZone-96156374, ID : 635e2a47-ae48-497f-9484-30afb609d66d
2025-09-11 03:19:06,875 - DEBUG - Services list from nova: [, , , ]
2025-09-11 03:19:06,875 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4079-1', 'an-jenkins-deploy-platform-ansible-os-4079-2']}
2025-09-11 03:19:06,877 - INFO - ================================================================================
2025-09-11 03:19:06,877 - INFO - STARTING TEST : test_bgp_control_node_zone
2025-09-11 03:19:08,140 - DEBUG - Skipping xmpp flap check
2025-09-11 03:19:08,140 - INFO - Initial checks done. Running the testcase now
2025-09-11 03:19:08,140 - INFO -
2025-09-11 03:19:08,140 - INFO - executing bgp_control_zone test
2025-09-11 03:19:08,861 - DEBUG - Response for create_network : {'network': {'id': 'e7f4137c-b87b-423a-a00c-9b92f59cd65e', 'name': 'ctest-cnz_vn-20373915', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'project_id': '635e2a47ae48497f948430afb609d66d', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-96156374', 'ctest-cnz_vn-20373915'], 'port_security_enabled': True, 'description': ''}}
2025-09-11 03:19:09,113 - DEBUG - Response for create_subnet : {'subnet': {'id': '98b0edae-abe4-419f-ac0a-d4028193024d', 'name': '', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'network_id': 'e7f4137c-b87b-423a-a00c-9b92f59cd65e', 'ip_version': 4, 'cidr': '199.39.46.192/26', 'allocation_pools': [{'start': '199.39.46.194', 'end': '199.39.46.254'}], 'gateway_ip': '199.39.46.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '199.39.46.194', 'tags': [], 'project_id': '635e2a47ae48497f948430afb609d66d'}}
2025-09-11 03:19:09,134 - INFO - Created VN ctest-cnz_vn-20373915
2025-09-11 03:19:09,146 - DEBUG - VN ctest-cnz_vn-20373915 UUID is e7f4137c-b87b-423a-a00c-9b92f59cd65e
2025-09-11 03:19:10,616 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1)
2025-09-11 03:19:12,294 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2)
2025-09-11 03:19:14,197 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1)
2025-09-11 03:19:14,197 - INFO - Waiting for VM ctest-TestBGPaasZone-96156374-09243738 to be up..
2025-09-11 03:19:14,307 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-09-11 03:19:19,396 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-09-11 03:19:24,484 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-09-11 03:19:29,567 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-09-11 03:19:34,657 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-09-11 03:19:39,741 - DEBUG - VM is still in BUILD state, Expected: ACTIVE
2025-09-11 03:19:44,824 - DEBUG - VM is in ACTIVE state now
2025-09-11 03:19:44,824 - INFO - VM name : ctest-TestBGPaasZone-96156374-09243738
2025-09-11 03:19:44,916 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 ID is 7d824a6d-0c2b-41d3-84ff-4ce7d51808f9
2025-09-11 03:19:44,952 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 launched on Node an-jenkins-deploy-platform-ansible-os-4079-1
2025-09-11 03:19:45,035 - DEBUG - Requesting: http://10.0.0.49:8082/virtual-machine/7d824a6d-0c2b-41d3-84ff-4ce7d51808f9
2025-09-11 03:19:45,344 - DEBUG - Requesting: http://10.0.0.49:8082/virtual-machine/7d824a6d-0c2b-41d3-84ff-4ce7d51808f9
2025-09-11 03:19:45,378 - DEBUG - Requesting: http://10.0.0.49:8082/virtual-machine-interface/b0a40c03-9afb-499c-aa79-90f42ad2fe14
2025-09-11 03:19:48,553 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms')
2025-09-11 03:19:48,553 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:19:48,570 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:19:52,654 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms')
2025-09-11 03:19:52,655 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:19:52,669 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:19:56,750 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms')
2025-09-11 03:19:56,750 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:19:56,766 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:00,848 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms')
2025-09-11 03:20:00,848 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:00,864 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:04,941 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms')
2025-09-11 03:20:04,941 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:04,956 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:09,033 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms')
2025-09-11 03:20:09,033 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:09,047 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:13,130 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1026ms')
2025-09-11 03:20:13,130 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:13,145 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:17,232 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms')
2025-09-11 03:20:17,232 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:17,248 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:21,322 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-09-11 03:20:21,323 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:21,342 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:25,419 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms')
2025-09-11 03:20:25,419 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:25,433 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:29,522 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms')
2025-09-11 03:20:29,522 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:29,539 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:33,613 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms')
2025-09-11 03:20:33,613 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:33,628 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:37,710 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms')
2025-09-11 03:20:37,710 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:37,728 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:41,808 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms')
2025-09-11 03:20:41,809 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:41,822 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:45,900 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-09-11 03:20:45,900 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:45,916 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:50,002 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms')
2025-09-11 03:20:50,002 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:50,017 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:54,122 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms')
2025-09-11 03:20:54,123 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:54,138 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:20:58,223 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms')
2025-09-11 03:20:58,224 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:20:58,240 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:21:02,316 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms')
2025-09-11 03:21:02,316 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:21:02,331 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:21:06,412 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms')
2025-09-11 03:21:06,413 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed!
2025-09-11 03:21:06,429 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set
2025-09-11 03:21:08,500 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n64 bytes from 169.254.0.7: icmp_seq=1 ttl=63 time=4.74 ms\r\n64 bytes from 169.254.0.7: icmp_seq=2 ttl=63 time=1.99 ms\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.988/3.364/4.741/1.376 ms')
2025-09-11 03:21:08,501 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 passed
2025-09-11 03:21:08,567 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:08,568 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:08,644 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:08,731 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:13,731 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:13,731 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:13,801 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:13,884 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:18,884 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:18,885 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:18,953 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:19,035 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:24,036 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:24,036 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:24,095 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:24,189 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:29,190 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:29,190 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:29,257 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:29,344 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:34,345 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:34,345 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:34,413 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:34,489 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:39,489 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:39,490 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:39,557 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:39,645 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:44,646 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:44,647 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:44,707 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:44,788 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:49,788 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:49,788 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:49,847 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:49,932 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:21:54,933 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:21:54,933 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:21:54,991 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-09-11 03:21:55,066 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE
2025-09-11 03:22:00,067 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-09-11 03:22:00,067 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22
2025-09-11 03:22:00,240 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is ready for SSH connections
2025-09-11 03:22:00,241 - INFO - Waiting for VM ctest-TestBGPaasZone-96156374-45803196 to be up..
2025-09-11 03:22:00,312 - DEBUG - VM is in ERROR state now
2025-09-11 03:22:00,312 - ERROR - VM in error state.
2025-09-11 03:22:00,312 - ERROR - VM is not in ACTIVE state
2025-09-11 03:22:00,402 - ERROR - Fatal Nova Exception while getting VM detail
2025-09-11 03:22:00,402 - DEBUG - None
2025-09-11 03:22:00,402 - ERROR - VM ctest-TestBGPaasZone-96156374-45803196 has not booted
2025-09-11 03:22:00,402 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-09243738
2025-09-11 03:22:00,478 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-45803196 in state ERROR
2025-09-11 03:22:00,479 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-45803196
2025-09-11 03:22:00,479 - ERROR - VM ctest-TestBGPaasZone-96156374-45803196 has failed to come up
2025-09-11 03:22:00,479 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:00,483 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-91608722 in state BUILD
2025-09-11 03:22:00,483 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-91608722
2025-09-11 03:22:00,484 - ERROR - VM ctest-TestBGPaasZone-96156374-91608722 has failed to come up
2025-09-11 03:22:00,484 - ERROR - Nova failed to get host of the VM
2025-09-11 03:22:00,484 - ERROR - VM ctest-TestBGPaasZone-96156374-91608722 has failed to come up
2025-09-11 03:22:00,484 - ERROR - Nova failed to get host of the VM
2025-09-11 03:22:00,566 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-45803196 in state ERROR
2025-09-11 03:22:00,566 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-45803196
2025-09-11 03:22:00,566 - ERROR - VM ctest-TestBGPaasZone-96156374-45803196 has failed to come up
2025-09-11 03:22:00,566 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:00,567 - INFO - Deleting VN ctest-cnz_vn-20373915
2025-09-11 03:22:00,612 - DEBUG - VN e7f4137c-b87b-423a-a00c-9b92f59cd65e still in use: Unable to complete operation on network e7f4137c-b87b-423a-a00c-9b92f59cd65e. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-0f5d35ae-b9b2-4703-8f32-48a9e299d053']
2025-09-11 03:22:00,613 - WARNING - Deleting VN ctest-cnz_vn-20373915 failed..Will retry
2025-09-11 03:22:02,798 - DEBUG - Response for deleting network ()
2025-09-11 03:22:04,076 - ERROR - AssertionError
Python 3.9.21: /usr/bin/python3
Thu Sep 11 03:22:00 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in test_bgp_control_node_zone(self=)
57 cnt = 0
58 self.logger.info('executing bgp_control_zone test')
59 assert self.basic_control_node_zone_setup()
60 # update control node zone with different bgp routers
61 self.update_control_node_zones(self.cnzs)
self =
self.basic_control_node_zone_setup = >
/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in basic_control_node_zone_setup(self=, ctrl_zone=None)
31 self.vms.append(self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird'))
32 self.client_vm = self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird')
33 self.check_vms_booted(self.vms + [self.client_vm])
34 for bgp_name in self.inputs.bgp_names:
35 self.cnzs += self.create_control_node_zones("test-zone",[bgp_name])
self =
self.check_vms_booted = >
self.vms = [, ]
self.client_vm =
/contrail-test/common/base.py in check_vms_booted(cls=, vms_list=[, , ], do_assert=True)
719 if failed and do_assert:
720 for vm_fixture in vms_list:
721 vm_fixture.cleanUp()
722 assert False, 'One or more vm-boots failed. Check logs'
723 if failed:
vm_fixture =
vm_fixture.cleanUp = >
/contrail-test/fixtures/vm_test.py in cleanUp(self=)
2050
2051 def cleanUp(self):
2052 self.delete()
2053 super(VMFixture, self).cleanUp()
2054
self =
self.delete = >
/contrail-test/fixtures/vm_test.py in delete(self=, verify=False, force=False)
2080 self.webui.delete_vm(self)
2081 else:
2082 self.orch.delete_vm(vm_obj, force=force)
2083 self.vm_objs.remove(vm_obj)
2084 if self.inputs.ns_agilio_vrouter_data:
self =
self.orch =
self.orch.delete_vm = >
vm_obj =
force = False
/contrail-test/fixtures/openstack.py in delete_vm(self=, vm_obj=, **kwargs={'force': False})
119
120 def delete_vm(self, vm_obj, **kwargs):
121 return self.nova_h.delete_vm(vm_obj, **kwargs)
122
123 def is_vm_deleted(self, vm_obj, **kwargs):
self =
self.nova_h =
self.nova_h.delete_vm = >
vm_obj =
kwargs = {'force': False}
/contrail-test/fixtures/nova_test.py in delete_vm(self=, vm_obj=, force=False)
763 def delete_vm(self, vm_obj, force=False):
764 if not force:
765 self.get_nova_host_of_vm(vm_obj)
766 vm_obj.delete()
767 # end _delete_vm
self =
self.get_nova_host_of_vm = >
vm_obj =
/contrail-test/fixtures/nova_test.py in get_nova_host_of_vm(self=, vm_obj=)
734 self.logger.error('VM %s has failed to come up' % vm_obj.name)
735 self.logger.error('Fault seen in nova show is: %s' % vm_obj.__dict__['fault'])
736 assert False, 'Fault seen in nova show %s is: %s' % (
737 vm_obj.id, vm_obj.__dict__['fault'])
738 else:
vm_obj =
vm_obj.id = 'c6f6ecd9-ba12-4220-9c56-fce14bbd1bf8'
vm_obj.__dict__ = {'OS-DCF:diskConfig': 'MANUAL', 'OS-EXT-AZ:availability_zone': 'nova', 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-0000001a', 'OS-EXT-STS:power_state': 0, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, ...}
AssertionError: Fault seen in nova show c6f6ecd9-ba12-4220-9c56-fce14bbd1bf8 is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = 'Assertion failed.'
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
args = (r"Fault seen in nova show c6f6ecd9-ba12-4220-9c56-...tion.NoValidHost: No valid host was found. \n\n'}",)
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 59, in test_bgp_control_node_zone
assert self.basic_control_node_zone_setup()
File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 33, in basic_control_node_zone_setup
self.check_vms_booted(self.vms + [self.client_vm])
File "/contrail-test/common/base.py", line 721, in check_vms_booted
vm_fixture.cleanUp()
File "/contrail-test/fixtures/vm_test.py", line 2052, in cleanUp
self.delete()
File "/contrail-test/fixtures/vm_test.py", line 2082, in delete
self.orch.delete_vm(vm_obj, force=force)
File "/contrail-test/fixtures/openstack.py", line 121, in delete_vm
return self.nova_h.delete_vm(vm_obj, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 765, in delete_vm
self.get_nova_host_of_vm(vm_obj)
File "/contrail-test/fixtures/nova_test.py", line 736, in get_nova_host_of_vm
assert False, 'Fault seen in nova show %s is: %s' % (
AssertionError: Fault seen in nova show c6f6ecd9-ba12-4220-9c56-fce14bbd1bf8 is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:04,076 - DEBUG - Skipping xmpp flap check
2025-09-11 03:22:04,076 - INFO -
2025-09-11 03:22:04,076 - INFO - END TEST : test_bgp_control_node_zone : FAILED[0:02:58]
2025-09-11 03:22:04,076 - INFO - --------------------------------------------------------------------------------
2025-09-11 03:22:04,079 - INFO - ================================================================================
2025-09-11 03:22:04,079 - INFO - STARTING TEST : test_bgp_control_node_zones_from_single_vnf
2025-09-11 03:22:05,331 - DEBUG - Skipping xmpp flap check
2025-09-11 03:22:05,331 - INFO - Initial checks done. Running the testcase now
2025-09-11 03:22:05,331 - INFO -
2025-09-11 03:22:05,331 - INFO - executing bgp_control_zone agent restart test
2025-09-11 03:22:05,595 - DEBUG - Response for create_network : {'network': {'id': 'e11d74cb-ff91-46bd-b6d8-1aa9af673435', 'name': 'ctest-cnz_vn-86152394', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'project_id': '635e2a47ae48497f948430afb609d66d', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-96156374', 'ctest-cnz_vn-86152394'], 'port_security_enabled': True, 'description': ''}}
2025-09-11 03:22:05,831 - DEBUG - Response for create_subnet : {'subnet': {'id': '784886d3-5435-433f-b016-1589248ed946', 'name': '', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'network_id': 'e11d74cb-ff91-46bd-b6d8-1aa9af673435', 'ip_version': 4, 'cidr': '103.91.158.0/26', 'allocation_pools': [{'start': '103.91.158.2', 'end': '103.91.158.62'}], 'gateway_ip': '103.91.158.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '103.91.158.2', 'tags': [], 'project_id': '635e2a47ae48497f948430afb609d66d'}}
2025-09-11 03:22:05,853 - INFO - Created VN ctest-cnz_vn-86152394
2025-09-11 03:22:05,867 - DEBUG - VN ctest-cnz_vn-86152394 UUID is e11d74cb-ff91-46bd-b6d8-1aa9af673435
2025-09-11 03:22:07,459 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2)
2025-09-11 03:22:08,829 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1)
2025-09-11 03:22:10,371 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2)
2025-09-11 03:22:10,371 - INFO - Waiting for VM ctest-TestBGPaasZone-96156374-43159470 to be up..
2025-09-11 03:22:10,429 - DEBUG - VM is in ERROR state now
2025-09-11 03:22:10,430 - ERROR - VM in error state.
2025-09-11 03:22:10,430 - ERROR - VM is not in ACTIVE state
2025-09-11 03:22:10,515 - ERROR - Fatal Nova Exception while getting VM detail
2025-09-11 03:22:10,515 - DEBUG - None
2025-09-11 03:22:10,516 - ERROR - VM ctest-TestBGPaasZone-96156374-43159470 has not booted
2025-09-11 03:22:10,516 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-43159470 in state ERROR
2025-09-11 03:22:10,516 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-43159470
2025-09-11 03:22:10,516 - ERROR - VM ctest-TestBGPaasZone-96156374-43159470 has failed to come up
2025-09-11 03:22:10,516 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:10,519 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-05185860 in state ERROR
2025-09-11 03:22:10,519 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-05185860
2025-09-11 03:22:10,519 - ERROR - VM ctest-TestBGPaasZone-96156374-05185860 has failed to come up
2025-09-11 03:22:10,519 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:22:10Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:10,519 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-73388571 in state BUILD
2025-09-11 03:22:10,520 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-73388571
2025-09-11 03:22:10,520 - ERROR - VM ctest-TestBGPaasZone-96156374-73388571 has failed to come up
2025-09-11 03:22:10,520 - ERROR - Nova failed to get host of the VM
2025-09-11 03:22:10,520 - ERROR - VM ctest-TestBGPaasZone-96156374-73388571 has failed to come up
2025-09-11 03:22:10,520 - ERROR - Nova failed to get host of the VM
2025-09-11 03:22:10,593 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-43159470 in state ERROR
2025-09-11 03:22:10,593 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-43159470
2025-09-11 03:22:10,593 - ERROR - VM ctest-TestBGPaasZone-96156374-43159470 has failed to come up
2025-09-11 03:22:10,593 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:10,593 - INFO - Deleting VN ctest-cnz_vn-86152394
2025-09-11 03:22:10,639 - DEBUG - VN e11d74cb-ff91-46bd-b6d8-1aa9af673435 still in use: Unable to complete operation on network e11d74cb-ff91-46bd-b6d8-1aa9af673435. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-510bf467-ff87-4103-b253-9276f06f3a98']
2025-09-11 03:22:10,640 - WARNING - Deleting VN ctest-cnz_vn-86152394 failed..Will retry
2025-09-11 03:22:12,808 - DEBUG - Response for deleting network ()
2025-09-11 03:22:14,090 - ERROR - AssertionError
Python 3.9.21: /usr/bin/python3
Thu Sep 11 03:22:10 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in test_bgp_control_node_zones_from_single_vnf(self=)
145 self.logger.info('executing bgp_control_zone agent restart test')
146 host_ips = []
147 assert self.basic_control_node_zone_setup('both-zones')
148 for vm in self.vms:
149 assert self.verify_bgpaas_in_control_nodes_and_agent(self.bgpaas_fixtures[cnt],vm)
self =
self.basic_control_node_zone_setup = >
/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in basic_control_node_zone_setup(self=, ctrl_zone='both-zones')
31 self.vms.append(self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird'))
32 self.client_vm = self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird')
33 self.check_vms_booted(self.vms + [self.client_vm])
34 for bgp_name in self.inputs.bgp_names:
35 self.cnzs += self.create_control_node_zones("test-zone",[bgp_name])
self =
self.check_vms_booted = >
self.vms = [, ]
self.client_vm =
/contrail-test/common/base.py in check_vms_booted(cls=, vms_list=[, , ], do_assert=True)
719 if failed and do_assert:
720 for vm_fixture in vms_list:
721 vm_fixture.cleanUp()
722 assert False, 'One or more vm-boots failed. Check logs'
723 if failed:
vm_fixture =
vm_fixture.cleanUp = >
/contrail-test/fixtures/vm_test.py in cleanUp(self=)
2050
2051 def cleanUp(self):
2052 self.delete()
2053 super(VMFixture, self).cleanUp()
2054
self =
self.delete = >
/contrail-test/fixtures/vm_test.py in delete(self=, verify=False, force=False)
2080 self.webui.delete_vm(self)
2081 else:
2082 self.orch.delete_vm(vm_obj, force=force)
2083 self.vm_objs.remove(vm_obj)
2084 if self.inputs.ns_agilio_vrouter_data:
self =
self.orch =
self.orch.delete_vm = >
vm_obj =
force = False
/contrail-test/fixtures/openstack.py in delete_vm(self=, vm_obj=, **kwargs={'force': False})
119
120 def delete_vm(self, vm_obj, **kwargs):
121 return self.nova_h.delete_vm(vm_obj, **kwargs)
122
123 def is_vm_deleted(self, vm_obj, **kwargs):
self =
self.nova_h =
self.nova_h.delete_vm = >
vm_obj =
kwargs = {'force': False}
/contrail-test/fixtures/nova_test.py in delete_vm(self=, vm_obj=, force=False)
763 def delete_vm(self, vm_obj, force=False):
764 if not force:
765 self.get_nova_host_of_vm(vm_obj)
766 vm_obj.delete()
767 # end _delete_vm
self =
self.get_nova_host_of_vm = >
vm_obj =
/contrail-test/fixtures/nova_test.py in get_nova_host_of_vm(self=, vm_obj=)
734 self.logger.error('VM %s has failed to come up' % vm_obj.name)
735 self.logger.error('Fault seen in nova show is: %s' % vm_obj.__dict__['fault'])
736 assert False, 'Fault seen in nova show %s is: %s' % (
737 vm_obj.id, vm_obj.__dict__['fault'])
738 else:
vm_obj =
vm_obj.id = 'a9e70736-4614-47d5-bd12-ecb6f2acf177'
vm_obj.__dict__ = {'OS-DCF:diskConfig': 'MANUAL', 'OS-EXT-AZ:availability_zone': 'nova', 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-0000001b', 'OS-EXT-STS:power_state': 0, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, ...}
AssertionError: Fault seen in nova show a9e70736-4614-47d5-bd12-ecb6f2acf177 is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = 'Assertion failed.'
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
args = (r"Fault seen in nova show a9e70736-4614-47d5-bd12-...tion.NoValidHost: No valid host was found. \n\n'}",)
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 147, in test_bgp_control_node_zones_from_single_vnf
assert self.basic_control_node_zone_setup('both-zones')
File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 33, in basic_control_node_zone_setup
self.check_vms_booted(self.vms + [self.client_vm])
File "/contrail-test/common/base.py", line 721, in check_vms_booted
vm_fixture.cleanUp()
File "/contrail-test/fixtures/vm_test.py", line 2052, in cleanUp
self.delete()
File "/contrail-test/fixtures/vm_test.py", line 2082, in delete
self.orch.delete_vm(vm_obj, force=force)
File "/contrail-test/fixtures/openstack.py", line 121, in delete_vm
return self.nova_h.delete_vm(vm_obj, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 765, in delete_vm
self.get_nova_host_of_vm(vm_obj)
File "/contrail-test/fixtures/nova_test.py", line 736, in get_nova_host_of_vm
assert False, 'Fault seen in nova show %s is: %s' % (
AssertionError: Fault seen in nova show a9e70736-4614-47d5-bd12-ecb6f2acf177 is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-09-11 03:22:14,091 - DEBUG - Skipping xmpp flap check
2025-09-11 03:22:14,091 - INFO -
2025-09-11 03:22:14,091 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : FAILED[0:00:10]
2025-09-11 03:22:14,091 - INFO - --------------------------------------------------------------------------------
2025-09-11 03:22:14,857 - INFO - Deleted project: ctest-TestBGPaasZone-96156374, ID : 635e2a47-ae48-497f-9484-30afb609d66d