2025-11-06 03:56:11,676 - INFO - Domain Default found not creating 2025-11-06 03:56:11,995 - INFO - Project ctest-TestBGPaasZone-35488535 not found, creating it 2025-11-06 03:56:12,646 - INFO - Created Project:ctest-TestBGPaasZone-35488535, ID : c1635dfb-4661-4ee7-9699-4cd3e39aa980 2025-11-06 03:56:14,198 - DEBUG - Services list from nova: [, , , ] 2025-11-06 03:56:14,198 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4662-2', 'an-jenkins-deploy-platform-ansible-os-4662-1']} 2025-11-06 03:56:14,200 - INFO - ================================================================================ 2025-11-06 03:56:14,200 - INFO - STARTING TEST : test_bgp_control_node_zone 2025-11-06 03:56:15,491 - DEBUG - Skipping xmpp flap check 2025-11-06 03:56:15,491 - INFO - Initial checks done. Running the testcase now 2025-11-06 03:56:15,492 - INFO - 2025-11-06 03:56:15,492 - INFO - executing bgp_control_zone test 2025-11-06 03:56:16,314 - DEBUG - Response for create_network : {'network': {'id': 'ebef1e3f-6a30-4f71-bb61-25a09433efa0', 'name': 'ctest-cnz_vn-66149691', 'tenant_id': 'c1635dfb46614ee796994cd3e39aa980', 'project_id': 'c1635dfb46614ee796994cd3e39aa980', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-35488535', 'ctest-cnz_vn-66149691'], 'port_security_enabled': True, 'description': ''}} 2025-11-06 03:56:16,600 - DEBUG - Response for create_subnet : {'subnet': {'id': '2793f49f-b8e4-4d60-89fa-13a3bab1cda2', 'name': '', 'tenant_id': 'c1635dfb46614ee796994cd3e39aa980', 'network_id': 'ebef1e3f-6a30-4f71-bb61-25a09433efa0', 'ip_version': 4, 'cidr': '18.133.146.128/26', 'allocation_pools': [{'start': '18.133.146.130', 'end': '18.133.146.190'}], 'gateway_ip': '18.133.146.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '18.133.146.130', 'tags': [], 'project_id': 'c1635dfb46614ee796994cd3e39aa980'}} 2025-11-06 03:56:16,635 - INFO - Created VN ctest-cnz_vn-66149691 2025-11-06 03:56:16,657 - DEBUG - VN ctest-cnz_vn-66149691 UUID is ebef1e3f-6a30-4f71-bb61-25a09433efa0 2025-11-06 03:56:17,969 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4662-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4662-2) 2025-11-06 03:56:19,607 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4662-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4662-1) 2025-11-06 03:56:21,369 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4662-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4662-2) 2025-11-06 03:56:21,369 - INFO - Waiting for VM ctest-TestBGPaasZone-35488535-88202134 to be up.. 2025-11-06 03:56:21,491 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:26,578 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:31,665 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:36,758 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:41,840 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:46,926 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:52,027 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-06 03:56:57,112 - DEBUG - VM is in ACTIVE state now 2025-11-06 03:56:57,113 - INFO - VM name : ctest-TestBGPaasZone-35488535-88202134 2025-11-06 03:56:57,208 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 ID is 248e530f-07dd-43de-8d3c-82374fedebff 2025-11-06 03:56:57,234 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 launched on Node an-jenkins-deploy-platform-ansible-os-4662-2 2025-11-06 03:56:57,321 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/248e530f-07dd-43de-8d3c-82374fedebff 2025-11-06 03:56:57,624 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/248e530f-07dd-43de-8d3c-82374fedebff 2025-11-06 03:56:57,658 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/b0bff04d-10b5-4ce8-b926-6cdf919b9f18 2025-11-06 03:57:00,850 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1025ms') 2025-11-06 03:57:00,850 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:00,868 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:04,944 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-11-06 03:57:04,944 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:04,960 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:09,048 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms') 2025-11-06 03:57:09,048 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:09,067 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:13,142 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms') 2025-11-06 03:57:13,142 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:13,159 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:17,235 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-11-06 03:57:17,235 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:17,250 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:21,333 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-11-06 03:57:21,333 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:21,349 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:25,423 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-11-06 03:57:25,423 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:25,438 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:29,519 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-11-06 03:57:29,519 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:29,535 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:33,621 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-11-06 03:57:33,621 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:33,639 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:37,716 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-11-06 03:57:37,716 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:37,733 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:41,812 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-11-06 03:57:41,812 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:41,827 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:45,905 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-11-06 03:57:45,905 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:45,939 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:50,006 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1003ms') 2025-11-06 03:57:50,006 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:50,021 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:54,106 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-11-06 03:57:54,106 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:54,121 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:57:58,189 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms') 2025-11-06 03:57:58,189 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:57:58,206 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:58:02,284 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-11-06 03:58:02,284 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:58:02,309 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:58:06,381 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-11-06 03:58:06,381 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:58:06,395 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:58:10,485 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-11-06 03:58:10,486 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:58:10,500 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:58:14,578 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-11-06 03:58:14,578 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 failed! 2025-11-06 03:58:14,593 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-66149691 is 18.133.146.129 and allocation pool is NOT set 2025-11-06 03:58:16,654 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.77 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.97 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.968/3.866/5.765/1.898 ms') 2025-11-06 03:58:16,654 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-88202134 passed 2025-11-06 03:58:16,722 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:16,723 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:16,789 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:16,917 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:21,917 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:21,918 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:21,996 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:22,083 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:27,083 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:27,084 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:27,149 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:27,250 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:32,252 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:32,252 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:32,318 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:32,412 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:37,413 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:37,413 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:37,471 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:37,553 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:42,554 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:42,554 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:42,615 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:42,703 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:47,704 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:47,704 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:47,774 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:47,859 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:52,860 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:52,860 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:52,933 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:53,022 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:58:58,022 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:58:58,022 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:58:58,093 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:58:58,185 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:59:03,186 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:03,186 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:59:03,253 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:59:03,334 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:59:08,334 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:08,334 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:59:08,401 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 03:59:08,487 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 03:59:13,488 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:13,488 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-88202134, IP 18.133.146.131, Port 22 2025-11-06 03:59:13,661 - DEBUG - VM ctest-TestBGPaasZone-35488535-88202134 is ready for SSH connections 2025-11-06 03:59:13,661 - INFO - Waiting for VM ctest-TestBGPaasZone-35488535-93482730 to be up.. 2025-11-06 03:59:13,748 - DEBUG - VM is in ACTIVE state now 2025-11-06 03:59:13,748 - INFO - VM name : ctest-TestBGPaasZone-35488535-93482730 2025-11-06 03:59:13,833 - DEBUG - VM ctest-TestBGPaasZone-35488535-93482730 ID is 3ca78eb1-4855-44b7-a3cb-d176aaf91bfd 2025-11-06 03:59:13,833 - DEBUG - VM ctest-TestBGPaasZone-35488535-93482730 launched on Node an-jenkins-deploy-platform-ansible-os-4662-1 2025-11-06 03:59:13,916 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/3ca78eb1-4855-44b7-a3cb-d176aaf91bfd 2025-11-06 03:59:13,927 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/cf83772c-52d1-4e65-9a59-769489cdba89 2025-11-06 03:59:15,095 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=4.71 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.462 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.462/2.583/4.705/2.121 ms') 2025-11-06 03:59:15,095 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-93482730 passed 2025-11-06 03:59:15,167 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:15,167 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-93482730, IP 18.133.146.132, Port 22 2025-11-06 03:59:15,334 - DEBUG - VM ctest-TestBGPaasZone-35488535-93482730 is ready for SSH connections 2025-11-06 03:59:15,334 - INFO - Waiting for VM ctest-TestBGPaasZone-35488535-94122849 to be up.. 2025-11-06 03:59:15,417 - DEBUG - VM is in ACTIVE state now 2025-11-06 03:59:15,417 - INFO - VM name : ctest-TestBGPaasZone-35488535-94122849 2025-11-06 03:59:15,502 - DEBUG - VM ctest-TestBGPaasZone-35488535-94122849 ID is dfa98ca0-1236-4f03-8c1c-59042f61bcac 2025-11-06 03:59:15,502 - DEBUG - VM ctest-TestBGPaasZone-35488535-94122849 launched on Node an-jenkins-deploy-platform-ansible-os-4662-2 2025-11-06 03:59:15,587 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/dfa98ca0-1236-4f03-8c1c-59042f61bcac 2025-11-06 03:59:15,598 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/f74773b4-1b5f-4bb2-9d45-5d14df15fe0c 2025-11-06 03:59:16,782 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=163 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.778 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.778/82.129/163.480/81.351 ms') 2025-11-06 03:59:16,783 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBGPaasZone-35488535-94122849 passed 2025-11-06 03:59:16,856 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:16,857 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-94122849, IP 18.133.146.133, Port 22 2025-11-06 03:59:17,030 - DEBUG - VM ctest-TestBGPaasZone-35488535-94122849 is ready for SSH connections 2025-11-06 03:59:17,194 - INFO - Created control node zone ctest-test-zone-70829007(c378270f-c227-437d-bca3-fd8db43ce986) 2025-11-06 03:59:17,558 - INFO - Created control node zone ctest-test-zone-29718899(c1b7f5fc-2284-4cc6-9805-940380704066) 2025-11-06 03:59:18,562 - INFO - Created control node zone ctest-test-zone-25496830(9569ba1d-bf1c-4915-8a77-8e12ee48493f) 2025-11-06 03:59:18,846 - INFO - Creating BGPaaS ctest-TestBGPaasZone-35488535-42330780 2025-11-06 03:59:18,930 - INFO - BGPaaS: ctest-TestBGPaasZone-35488535-42330780(94ca151c-4c9f-406b-9951-700182000aab) 2025-11-06 03:59:18,931 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 03:59:19,227 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 03:59:19,267 - INFO - verify_in_api_server passed for BGPaaS obj 94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 03:59:19,267 - INFO - BGPaaS(94ca151c-4c9f-406b-9951-700182000aab): verify_on_setup passed 2025-11-06 03:59:19,267 - INFO - We will configure BGP on the VM 2025-11-06 03:59:19,546 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:19,546 - INFO - Configuring BGP on ctest-TestBGPaasZone-35488535-88202134 2025-11-06 03:59:19,547 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 03:59:19,547 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS 2025-11-06 03:59:33,635 - DEBUG - None 2025-11-06 03:59:33,635 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 03:59:33,636 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS 2025-11-06 03:59:34,751 - DEBUG - None 2025-11-06 03:59:34,751 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 03:59:34,751 - DEBUG - service bird restart 2025-11-06 03:59:38,160 - DEBUG - bird stop/waiting bird start/running, process 1439 2025-11-06 03:59:38,160 - INFO - Attaching VMI b0bff04d-10b5-4ce8-b926-6cdf919b9f18 to BGPaaS 94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 03:59:38,242 - INFO - Attaching the VMI 248e530f-07dd-43de-8d3c-82374fedebff to the BGPaaS 94ca151c-4c9f-406b-9951-700182000aab object 2025-11-06 03:59:38,242 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 18.133.146.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 03:59:38,242 - DEBUG - sudo ip addr add 18.133.146.148 dev eth0 2025-11-06 03:59:39,049 - DEBUG - None 2025-11-06 03:59:39,118 - INFO - Creating BGPaaS ctest-TestBGPaasZone-35488535-40055287 2025-11-06 03:59:39,198 - INFO - BGPaaS: ctest-TestBGPaasZone-35488535-40055287(ce197c07-4b04-4bf9-bb6b-22854d1e558f) 2025-11-06 03:59:39,199 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/ce197c07-4b04-4bf9-bb6b-22854d1e558f 2025-11-06 03:59:39,212 - INFO - verify_in_api_server passed for BGPaaS obj ce197c07-4b04-4bf9-bb6b-22854d1e558f 2025-11-06 03:59:39,212 - INFO - BGPaaS(ce197c07-4b04-4bf9-bb6b-22854d1e558f): verify_on_setup passed 2025-11-06 03:59:39,213 - INFO - We will configure BGP on the VM 2025-11-06 03:59:39,498 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:39,498 - INFO - Configuring BGP on ctest-TestBGPaasZone-35488535-93482730 2025-11-06 03:59:39,498 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.132; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.132; } protocol bfd { neighbor 18.133.146.130 local 18.133.146.132 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 03:59:39,499 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.132; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.132; } protocol bfd { neighbor 18.133.146.130 local 18.133.146.132 multihop on; } EOS 2025-11-06 03:59:45,290 - DEBUG - None 2025-11-06 03:59:45,290 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.132; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.132; } protocol bfd { neighbor 18.133.146.130 local 18.133.146.132 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 03:59:45,290 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.132; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.132; } protocol bfd { neighbor 18.133.146.130 local 18.133.146.132 multihop on; } EOS 2025-11-06 03:59:45,817 - DEBUG - None 2025-11-06 03:59:45,817 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 03:59:45,817 - DEBUG - service bird restart 2025-11-06 03:59:47,724 - DEBUG - bird stop/waiting bird start/running, process 1438 2025-11-06 03:59:47,724 - INFO - Attaching VMI cf83772c-52d1-4e65-9a59-769489cdba89 to BGPaaS ce197c07-4b04-4bf9-bb6b-22854d1e558f 2025-11-06 03:59:47,802 - INFO - Attaching the VMI 3ca78eb1-4855-44b7-a3cb-d176aaf91bfd to the BGPaaS ce197c07-4b04-4bf9-bb6b-22854d1e558f object 2025-11-06 03:59:47,802 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 18.133.146.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 03:59:47,802 - DEBUG - sudo ip addr add 18.133.146.148 dev eth0 2025-11-06 03:59:48,505 - DEBUG - None 2025-11-06 03:59:48,505 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:48,505 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 03:59:48,533 - INFO - BGPaaS session seen in control-node 10.20.0.200 2025-11-06 03:59:48,541 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 03:59:48,561 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.252'] 2025-11-06 03:59:54,579 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.252'] 2025-11-06 04:00:00,601 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:00:00,607 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 18.133.146.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:00,607 - DEBUG - ping -s 56 -c 3 -W 1 18.133.146.148 2025-11-06 04:00:07,258 - DEBUG - PING 18.133.146.148 (18.133.146.148) 56(84) bytes of data. 64 bytes from 18.133.146.148: icmp_seq=1 ttl=64 time=8.27 ms 64 bytes from 18.133.146.148: icmp_seq=2 ttl=64 time=1.49 ms 64 bytes from 18.133.146.148: icmp_seq=3 ttl=64 time=1.31 ms --- 18.133.146.148 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.310/3.694/8.278/3.242 ms 2025-11-06 04:00:07,258 - INFO - Ping to IP 18.133.146.148 from VM ctest-TestBGPaasZone-35488535-94122849 passed 2025-11-06 04:00:08,142 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:08,142 - DEBUG - service bird restart 2025-11-06 04:00:09,971 - DEBUG - bird stop/waiting bird start/running, process 1467 2025-11-06 04:00:09,971 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:00:09,971 - DEBUG - service bird restart 2025-11-06 04:00:11,835 - DEBUG - bird stop/waiting bird start/running, process 1465 2025-11-06 04:00:11,835 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:00:11,854 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.249'] 2025-11-06 04:00:17,874 - INFO - BGPaaS session seen in control-node 10.20.0.249 2025-11-06 04:00:17,880 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:00:17,898 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:00:17,905 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 18.133.146.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:17,905 - DEBUG - ping -s 56 -c 3 -W 1 18.133.146.148 2025-11-06 04:00:20,423 - DEBUG - PING 18.133.146.148 (18.133.146.148) 56(84) bytes of data. 64 bytes from 18.133.146.148: icmp_seq=1 ttl=64 time=3.96 ms 64 bytes from 18.133.146.148: icmp_seq=2 ttl=64 time=1.07 ms 64 bytes from 18.133.146.148: icmp_seq=3 ttl=64 time=1.71 ms --- 18.133.146.148 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.072/2.253/3.969/1.242 ms 2025-11-06 04:00:20,423 - INFO - Ping to IP 18.133.146.148 from VM ctest-TestBGPaasZone-35488535-94122849 passed 2025-11-06 04:00:20,743 - INFO - We will configure BGP on the VM 2025-11-06 04:00:20,966 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:00:20,966 - INFO - Configuring BGP on ctest-TestBGPaasZone-35488535-88202134 2025-11-06 04:00:20,967 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:20,967 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS 2025-11-06 04:00:21,473 - DEBUG - None 2025-11-06 04:00:21,474 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:21,474 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 18.133.146.131; protocol bgp bgp_1{ local as 65000; neighbor 18.133.146.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 18.133.146.131; } protocol bfd { neighbor 18.133.146.129 local 18.133.146.131 multihop on; } EOS 2025-11-06 04:00:22,057 - DEBUG - None 2025-11-06 04:00:22,057 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:22,057 - DEBUG - service bird restart 2025-11-06 04:00:23,957 - DEBUG - bird stop/waiting bird start/running, process 1499 2025-11-06 04:00:23,958 - INFO - Attaching VMI b0bff04d-10b5-4ce8-b926-6cdf919b9f18 to BGPaaS 94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 04:00:23,980 - INFO - Attaching the VMI 248e530f-07dd-43de-8d3c-82374fedebff to the BGPaaS 94ca151c-4c9f-406b-9951-700182000aab object 2025-11-06 04:00:23,980 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 18.133.146.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:23,980 - DEBUG - sudo ip addr add 18.133.146.148 dev eth0 2025-11-06 04:00:24,860 - DEBUG - RTNETLINK answers: File exists 2025-11-06 04:00:24,860 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:24,860 - DEBUG - service bird restart 2025-11-06 04:00:26,878 - DEBUG - bird stop/waiting bird start/running, process 1526 2025-11-06 04:00:26,878 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:00:26,878 - DEBUG - service bird restart 2025-11-06 04:00:28,695 - DEBUG - bird stop/waiting bird start/running, process 1487 2025-11-06 04:00:28,695 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:00:28,695 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:00:28,711 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.200'] 2025-11-06 04:00:34,713 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:00:34,730 - INFO - BGPaaS session seen in control-node 10.20.0.200 2025-11-06 04:00:34,735 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:00:34,750 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:00:34,757 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 18.133.146.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:00:34,757 - DEBUG - ping -s 56 -c 3 -W 1 18.133.146.148 2025-11-06 04:00:37,266 - DEBUG - PING 18.133.146.148 (18.133.146.148) 56(84) bytes of data. 64 bytes from 18.133.146.148: icmp_seq=1 ttl=64 time=1.11 ms 64 bytes from 18.133.146.148: icmp_seq=2 ttl=64 time=1.12 ms 64 bytes from 18.133.146.148: icmp_seq=3 ttl=64 time=1.12 ms --- 18.133.146.148 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.115/1.122/1.129/0.005 ms 2025-11-06 04:00:37,267 - INFO - Ping to IP 18.133.146.148 from VM ctest-TestBGPaasZone-35488535-94122849 passed 2025-11-06 04:00:37,267 - INFO - Detaching VMI b0bff04d-10b5-4ce8-b926-6cdf919b9f18 from BGPaaS 94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 04:00:37,342 - INFO - Detaching VMI cf83772c-52d1-4e65-9a59-769489cdba89 from BGPaaS ce197c07-4b04-4bf9-bb6b-22854d1e558f 2025-11-06 04:00:37,470 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-35488535-40055287(ce197c07-4b04-4bf9-bb6b-22854d1e558f) 2025-11-06 04:00:37,553 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/ce197c07-4b04-4bf9-bb6b-22854d1e558f 2025-11-06 04:00:37,562 - DEBUG - Response Code: 404 2025-11-06 04:00:37,562 - DEBUG - BGPaaS: ce197c07-4b04-4bf9-bb6b-22854d1e558f deleted from api server 2025-11-06 04:00:37,562 - INFO - BGPaaS(ce197c07-4b04-4bf9-bb6b-22854d1e558f): verify_on_cleanup passed 2025-11-06 04:00:37,562 - INFO - Detaching VMI b0bff04d-10b5-4ce8-b926-6cdf919b9f18 from BGPaaS 94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 04:00:37,603 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-35488535-42330780(94ca151c-4c9f-406b-9951-700182000aab) 2025-11-06 04:00:37,691 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/94ca151c-4c9f-406b-9951-700182000aab 2025-11-06 04:00:37,698 - DEBUG - Response Code: 404 2025-11-06 04:00:37,698 - DEBUG - BGPaaS: 94ca151c-4c9f-406b-9951-700182000aab deleted from api server 2025-11-06 04:00:37,698 - INFO - BGPaaS(94ca151c-4c9f-406b-9951-700182000aab): verify_on_cleanup passed 2025-11-06 04:00:37,698 - INFO - Deleting Control node zone ctest-test-zone-25496830(9569ba1d-bf1c-4915-8a77-8e12ee48493f) 2025-11-06 04:00:37,854 - INFO - Deleting Control node zone ctest-test-zone-29718899(c1b7f5fc-2284-4cc6-9805-940380704066) 2025-11-06 04:00:38,010 - INFO - Deleting Control node zone ctest-test-zone-70829007(c378270f-c227-437d-bca3-fd8db43ce986) 2025-11-06 04:00:38,093 - INFO - Deleting VM ctest-TestBGPaasZone-35488535-94122849 2025-11-06 04:00:38,164 - INFO - Deleting VM ctest-TestBGPaasZone-35488535-93482730 2025-11-06 04:00:38,232 - INFO - Deleting VM ctest-TestBGPaasZone-35488535-88202134 2025-11-06 04:00:38,320 - INFO - Deleting VN ctest-cnz_vn-66149691 2025-11-06 04:00:38,435 - DEBUG - VN ebef1e3f-6a30-4f71-bb61-25a09433efa0 still in use: Unable to complete operation on network ebef1e3f-6a30-4f71-bb61-25a09433efa0. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-5ff09384-9cfa-4ed2-9062-c867536f2a43'] 2025-11-06 04:00:38,435 - WARNING - Deleting VN ctest-cnz_vn-66149691 failed..Will retry 2025-11-06 04:00:40,717 - DEBUG - Response for deleting network () 2025-11-06 04:00:42,007 - DEBUG - Skipping xmpp flap check 2025-11-06 04:00:42,007 - INFO - END TEST : test_bgp_control_node_zone : PASSED[0:04:28] 2025-11-06 04:00:42,007 - INFO - -------------------------------------------------------------------------------- 2025-11-06 04:00:42,011 - INFO - ================================================================================ 2025-11-06 04:00:42,011 - INFO - STARTING TEST : test_bgp_control_node_zones_from_single_vnf 2025-11-06 04:00:43,279 - DEBUG - Skipping xmpp flap check 2025-11-06 04:00:43,279 - INFO - Initial checks done. Running the testcase now 2025-11-06 04:00:43,279 - INFO - 2025-11-06 04:00:43,279 - INFO - executing bgp_control_zone agent restart test 2025-11-06 04:00:43,591 - DEBUG - Response for create_network : {'network': {'id': '5dfbb401-80d4-4253-9008-4d92357f50dd', 'name': 'ctest-cnz_vn-28236348', 'tenant_id': 'c1635dfb46614ee796994cd3e39aa980', 'project_id': 'c1635dfb46614ee796994cd3e39aa980', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-35488535', 'ctest-cnz_vn-28236348'], 'port_security_enabled': True, 'description': ''}} 2025-11-06 04:00:43,816 - DEBUG - Response for create_subnet : {'subnet': {'id': '259fe441-b69d-42f8-8348-e4a163984c88', 'name': '', 'tenant_id': 'c1635dfb46614ee796994cd3e39aa980', 'network_id': '5dfbb401-80d4-4253-9008-4d92357f50dd', 'ip_version': 4, 'cidr': '24.237.15.0/26', 'allocation_pools': [{'start': '24.237.15.2', 'end': '24.237.15.62'}], 'gateway_ip': '24.237.15.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '24.237.15.2', 'tags': [], 'project_id': 'c1635dfb46614ee796994cd3e39aa980'}} 2025-11-06 04:00:43,839 - INFO - Created VN ctest-cnz_vn-28236348 2025-11-06 04:00:43,859 - DEBUG - VN ctest-cnz_vn-28236348 UUID is 5dfbb401-80d4-4253-9008-4d92357f50dd 2025-11-06 04:00:45,053 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4662-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4662-1) 2025-11-06 04:00:46,519 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4662-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4662-2) 2025-11-06 04:00:48,127 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4662-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4662-1) 2025-11-06 04:00:48,128 - INFO - Waiting for VM ctest-TestBGPaasZone-35488535-99159074 to be up.. 2025-11-06 04:00:48,256 - DEBUG - VM is in ACTIVE state now 2025-11-06 04:00:48,256 - INFO - VM name : ctest-TestBGPaasZone-35488535-99159074 2025-11-06 04:00:48,364 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 ID is 701e48e5-3cee-49c8-9943-4c944fe307e6 2025-11-06 04:00:48,364 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 launched on Node an-jenkins-deploy-platform-ansible-os-4662-1 2025-11-06 04:00:48,465 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/701e48e5-3cee-49c8-9943-4c944fe307e6 2025-11-06 04:00:48,480 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/69761f1d-7a05-4a94-9c59-d900c9921681 2025-11-06 04:00:51,715 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1004ms') 2025-11-06 04:00:51,715 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:00:51,730 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:00:55,799 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1001ms') 2025-11-06 04:00:55,800 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:00:55,814 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:00:59,899 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-06 04:00:59,899 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:00:59,916 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:03,993 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-11-06 04:01:03,993 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:04,011 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:08,091 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-06 04:01:08,091 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:08,107 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:12,185 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-11-06 04:01:12,186 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:12,205 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:16,278 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-11-06 04:01:16,278 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:16,323 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:20,414 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-11-06 04:01:20,414 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:20,431 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:24,505 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-11-06 04:01:24,505 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:24,521 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:28,604 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-11-06 04:01:28,604 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:28,623 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:32,696 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-11-06 04:01:32,697 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:32,714 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:36,790 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-11-06 04:01:36,790 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:36,805 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:40,891 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-11-06 04:01:40,891 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:40,905 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:44,982 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-11-06 04:01:44,982 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:45,000 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:49,081 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-06 04:01:49,081 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:49,096 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:53,181 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-11-06 04:01:53,181 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:53,198 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:01:57,269 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-11-06 04:01:57,270 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:01:57,288 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:02:01,372 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-11-06 04:02:01,373 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:02:01,393 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:02:05,461 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1002ms') 2025-11-06 04:02:05,461 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 failed! 2025-11-06 04:02:05,476 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-35488535:ctest-cnz_vn-28236348 is 24.237.15.1 and allocation pool is NOT set 2025-11-06 04:02:07,544 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=2.61 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.26 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 2.614/2.937/3.260/0.323 ms') 2025-11-06 04:02:07,544 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-99159074 passed 2025-11-06 04:02:07,615 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:07,615 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:07,683 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:07,841 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:12,842 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:12,842 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:12,914 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:13,026 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:18,026 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:18,027 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:18,094 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:18,206 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:23,207 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:23,208 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:23,263 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:23,370 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:28,370 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:28,371 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:28,439 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:28,537 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:33,538 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:33,539 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:33,596 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:33,692 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:38,693 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:38,693 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:38,762 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:38,854 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:43,855 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:43,855 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:43,922 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:44,011 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:49,012 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:49,012 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:49,078 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:49,170 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:02:54,171 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:54,171 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-99159074, IP 24.237.15.3, Port 22 2025-11-06 04:02:54,342 - DEBUG - VM ctest-TestBGPaasZone-35488535-99159074 is ready for SSH connections 2025-11-06 04:02:54,342 - INFO - Waiting for VM ctest-TestBGPaasZone-35488535-59204111 to be up.. 2025-11-06 04:02:54,424 - DEBUG - VM is in ACTIVE state now 2025-11-06 04:02:54,424 - INFO - VM name : ctest-TestBGPaasZone-35488535-59204111 2025-11-06 04:02:54,512 - DEBUG - VM ctest-TestBGPaasZone-35488535-59204111 ID is 8d1691d6-c356-4a8e-8a06-cbd026c80ec0 2025-11-06 04:02:54,512 - DEBUG - VM ctest-TestBGPaasZone-35488535-59204111 launched on Node an-jenkins-deploy-platform-ansible-os-4662-2 2025-11-06 04:02:54,598 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/8d1691d6-c356-4a8e-8a06-cbd026c80ec0 2025-11-06 04:02:54,639 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/b8d6f665-a29d-4b7c-8518-7e39bb9a749a 2025-11-06 04:02:55,814 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.33 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.848 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.848/2.090/3.332/1.242 ms') 2025-11-06 04:02:55,814 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-35488535-59204111 passed 2025-11-06 04:02:55,888 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:55,888 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-59204111, IP 24.237.15.4, Port 22 2025-11-06 04:02:56,061 - DEBUG - VM ctest-TestBGPaasZone-35488535-59204111 is ready for SSH connections 2025-11-06 04:02:56,061 - INFO - Waiting for VM ctest-TestBGPaasZone-35488535-27888714 to be up.. 2025-11-06 04:02:56,147 - DEBUG - VM is in ACTIVE state now 2025-11-06 04:02:56,147 - INFO - VM name : ctest-TestBGPaasZone-35488535-27888714 2025-11-06 04:02:56,236 - DEBUG - VM ctest-TestBGPaasZone-35488535-27888714 ID is 7c2d25c6-4ca4-463e-985e-bf7044535436 2025-11-06 04:02:56,236 - DEBUG - VM ctest-TestBGPaasZone-35488535-27888714 launched on Node an-jenkins-deploy-platform-ansible-os-4662-1 2025-11-06 04:02:56,331 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/7c2d25c6-4ca4-463e-985e-bf7044535436 2025-11-06 04:02:56,341 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/bb02683c-10d2-4c0d-a545-78eaf8dc27e6 2025-11-06 04:02:57,494 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.96 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.741 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1000ms\r\nrtt min/avg/max/mdev = 0.741/2.352/3.964/1.611 ms') 2025-11-06 04:02:57,495 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBGPaasZone-35488535-27888714 passed 2025-11-06 04:02:57,577 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:02:57,577 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-27888714, IP 24.237.15.5, Port 22 2025-11-06 04:02:57,642 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-06 04:02:57,739 - DEBUG - VM ctest-TestBGPaasZone-35488535-27888714 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-06 04:03:02,739 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:02,739 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-35488535-27888714, IP 24.237.15.5, Port 22 2025-11-06 04:03:02,907 - DEBUG - VM ctest-TestBGPaasZone-35488535-27888714 is ready for SSH connections 2025-11-06 04:03:03,050 - INFO - Created control node zone ctest-test-zone-54210637(bafdc9f6-1633-488e-aa05-abfe7dcc3cc9) 2025-11-06 04:03:03,430 - INFO - Created control node zone ctest-test-zone-75920633(7869b2ef-39ec-4d15-9c9a-a9622eb3ce4c) 2025-11-06 04:03:03,806 - INFO - Created control node zone ctest-test-zone-33767907(48501058-834e-4727-b992-bc08c22b421a) 2025-11-06 04:03:04,118 - INFO - Creating BGPaaS ctest-TestBGPaasZone-35488535-86273212 2025-11-06 04:03:04,194 - INFO - BGPaaS: ctest-TestBGPaasZone-35488535-86273212(466e88b8-b228-43d6-9406-2a43a07240e4) 2025-11-06 04:03:04,194 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/466e88b8-b228-43d6-9406-2a43a07240e4 2025-11-06 04:03:04,205 - INFO - verify_in_api_server passed for BGPaaS obj 466e88b8-b228-43d6-9406-2a43a07240e4 2025-11-06 04:03:04,205 - INFO - BGPaaS(466e88b8-b228-43d6-9406-2a43a07240e4): verify_on_setup passed 2025-11-06 04:03:04,205 - INFO - We will configure BGP on the VM 2025-11-06 04:03:04,638 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:04,638 - INFO - Configuring BGP on ctest-TestBGPaasZone-35488535-99159074 2025-11-06 04:03:04,639 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 24.237.15.3; protocol bgp bgp_1{ local as 65000; neighbor 24.237.15.1 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.3; } protocol bfd { neighbor 24.237.15.1 local 24.237.15.3 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:03:04,639 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 24.237.15.3; protocol bgp bgp_1{ local as 65000; neighbor 24.237.15.1 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.3; } protocol bfd { neighbor 24.237.15.1 local 24.237.15.3 multihop on; } EOS 2025-11-06 04:03:18,003 - DEBUG - None 2025-11-06 04:03:18,003 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 24.237.15.2 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.3; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:03:18,003 - DEBUG - cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 24.237.15.2 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.3; } EOS 2025-11-06 04:03:18,550 - DEBUG - None 2025-11-06 04:03:18,550 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:03:18,551 - DEBUG - service bird restart 2025-11-06 04:03:20,595 - DEBUG - bird stop/waiting bird start/running, process 1433 2025-11-06 04:03:20,595 - INFO - Attaching VMI 69761f1d-7a05-4a94-9c59-d900c9921681 to BGPaaS 466e88b8-b228-43d6-9406-2a43a07240e4 2025-11-06 04:03:20,674 - INFO - Attaching the VMI 701e48e5-3cee-49c8-9943-4c944fe307e6 to the BGPaaS 466e88b8-b228-43d6-9406-2a43a07240e4 object 2025-11-06 04:03:20,674 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 24.237.15.20 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:03:20,674 - DEBUG - sudo ip addr add 24.237.15.20 dev eth0 2025-11-06 04:03:21,436 - DEBUG - None 2025-11-06 04:03:21,498 - INFO - Creating BGPaaS ctest-TestBGPaasZone-35488535-83113538 2025-11-06 04:03:21,582 - INFO - BGPaaS: ctest-TestBGPaasZone-35488535-83113538(066bcb2d-3a81-47b5-9b10-495e6cb2a43e) 2025-11-06 04:03:21,582 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/066bcb2d-3a81-47b5-9b10-495e6cb2a43e 2025-11-06 04:03:21,598 - INFO - verify_in_api_server passed for BGPaaS obj 066bcb2d-3a81-47b5-9b10-495e6cb2a43e 2025-11-06 04:03:21,598 - INFO - BGPaaS(066bcb2d-3a81-47b5-9b10-495e6cb2a43e): verify_on_setup passed 2025-11-06 04:03:21,598 - INFO - We will configure BGP on the VM 2025-11-06 04:03:22,046 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:22,046 - INFO - Configuring BGP on ctest-TestBGPaasZone-35488535-59204111 2025-11-06 04:03:22,047 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 24.237.15.4; protocol bgp bgp_1{ local as 65000; neighbor 24.237.15.1 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.4; } protocol bfd { neighbor 24.237.15.1 local 24.237.15.4 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:03:22,047 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 24.237.15.4; protocol bgp bgp_1{ local as 65000; neighbor 24.237.15.1 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.4; } protocol bfd { neighbor 24.237.15.1 local 24.237.15.4 multihop on; } EOS 2025-11-06 04:03:28,482 - DEBUG - None 2025-11-06 04:03:28,482 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 24.237.15.2 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.4; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:03:28,482 - DEBUG - cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 24.237.15.2 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 24.237.15.4; } EOS 2025-11-06 04:03:29,038 - DEBUG - None 2025-11-06 04:03:29,038 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:03:29,038 - DEBUG - service bird restart 2025-11-06 04:03:30,933 - DEBUG - bird stop/waiting bird start/running, process 1444 2025-11-06 04:03:30,933 - INFO - Attaching VMI b8d6f665-a29d-4b7c-8518-7e39bb9a749a to BGPaaS 066bcb2d-3a81-47b5-9b10-495e6cb2a43e 2025-11-06 04:03:31,022 - INFO - Attaching the VMI 8d1691d6-c356-4a8e-8a06-cbd026c80ec0 to the BGPaaS 066bcb2d-3a81-47b5-9b10-495e6cb2a43e object 2025-11-06 04:03:31,022 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 24.237.15.20 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2025-11-06 04:03:31,023 - DEBUG - sudo ip addr add 24.237.15.20 dev eth0 2025-11-06 04:03:31,839 - DEBUG - None 2025-11-06 04:03:31,839 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:31,839 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:03:31,857 - INFO - BGPaaS session seen in control-node 10.20.0.200 2025-11-06 04:03:31,857 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:31,876 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:03:31,884 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:31,884 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:03:31,898 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.200'] 2025-11-06 04:03:37,900 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:03:37,912 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.200'] 2025-11-06 04:03:43,913 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:03:43,925 - INFO - BGPaaS session seen in control-node 10.20.0.200 2025-11-06 04:03:43,925 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:43,943 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:03:43,948 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 24.237.15.20, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:03:43,948 - DEBUG - ping -s 56 -c 3 -W 1 24.237.15.20 2025-11-06 04:03:50,713 - DEBUG - PING 24.237.15.20 (24.237.15.20) 56(84) bytes of data. 64 bytes from 24.237.15.20: icmp_seq=1 ttl=64 time=8.53 ms 64 bytes from 24.237.15.20: icmp_seq=2 ttl=64 time=1.60 ms 64 bytes from 24.237.15.20: icmp_seq=3 ttl=64 time=1.30 ms --- 24.237.15.20 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.308/3.814/8.531/3.337 ms 2025-11-06 04:03:50,713 - INFO - Ping to IP 24.237.15.20 from VM ctest-TestBGPaasZone-35488535-27888714 passed 2025-11-06 04:03:50,713 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:50,713 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:03:50,729 - INFO - BGPaaS session seen in control-node 10.20.0.200 2025-11-06 04:03:50,730 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:50,746 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:03:50,751 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:50,751 - DEBUG - Requesting: http://10.20.0.200:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-06 04:03:50,772 - INFO - BGPaaS session seen in control-node 10.20.0.200 2025-11-06 04:03:50,773 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-06 04:03:50,795 - INFO - BGPaaS session seen in control-node 10.20.0.252 2025-11-06 04:03:50,804 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 24.237.15.20, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123 2025-11-06 04:03:50,804 - DEBUG - ping -s 56 -c 3 -W 1 24.237.15.20 2025-11-06 04:03:53,326 - DEBUG - PING 24.237.15.20 (24.237.15.20) 56(84) bytes of data. 64 bytes from 24.237.15.20: icmp_seq=1 ttl=64 time=10.4 ms 64 bytes from 24.237.15.20: icmp_seq=2 ttl=64 time=1.25 ms 64 bytes from 24.237.15.20: icmp_seq=3 ttl=64 time=1.35 ms --- 24.237.15.20 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.251/4.342/10.424/4.300 ms 2025-11-06 04:03:53,326 - INFO - Ping to IP 24.237.15.20 from VM ctest-TestBGPaasZone-35488535-27888714 passed 2025-11-06 04:03:53,326 - INFO - Detaching VMI b8d6f665-a29d-4b7c-8518-7e39bb9a749a from BGPaaS 066bcb2d-3a81-47b5-9b10-495e6cb2a43e 2025-11-06 04:03:53,423 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-35488535-83113538(066bcb2d-3a81-47b5-9b10-495e6cb2a43e) 2025-11-06 04:03:53,496 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/066bcb2d-3a81-47b5-9b10-495e6cb2a43e 2025-11-06 04:03:53,508 - DEBUG - Response Code: 404 2025-11-06 04:03:53,509 - DEBUG - BGPaaS: 066bcb2d-3a81-47b5-9b10-495e6cb2a43e deleted from api server 2025-11-06 04:03:53,509 - INFO - BGPaaS(066bcb2d-3a81-47b5-9b10-495e6cb2a43e): verify_on_cleanup passed 2025-11-06 04:03:53,509 - INFO - Detaching VMI 69761f1d-7a05-4a94-9c59-d900c9921681 from BGPaaS 466e88b8-b228-43d6-9406-2a43a07240e4 2025-11-06 04:03:53,634 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-35488535-86273212(466e88b8-b228-43d6-9406-2a43a07240e4) 2025-11-06 04:03:53,705 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/466e88b8-b228-43d6-9406-2a43a07240e4 2025-11-06 04:03:53,713 - DEBUG - Response Code: 404 2025-11-06 04:03:53,713 - DEBUG - BGPaaS: 466e88b8-b228-43d6-9406-2a43a07240e4 deleted from api server 2025-11-06 04:03:53,714 - INFO - BGPaaS(466e88b8-b228-43d6-9406-2a43a07240e4): verify_on_cleanup passed 2025-11-06 04:03:53,714 - INFO - Deleting Control node zone ctest-test-zone-33767907(48501058-834e-4727-b992-bc08c22b421a) 2025-11-06 04:03:53,879 - INFO - Deleting Control node zone ctest-test-zone-75920633(7869b2ef-39ec-4d15-9c9a-a9622eb3ce4c) 2025-11-06 04:03:54,039 - INFO - Deleting Control node zone ctest-test-zone-54210637(bafdc9f6-1633-488e-aa05-abfe7dcc3cc9) 2025-11-06 04:03:54,199 - INFO - Deleting VM ctest-TestBGPaasZone-35488535-27888714 2025-11-06 04:03:54,276 - INFO - Deleting VM ctest-TestBGPaasZone-35488535-59204111 2025-11-06 04:03:54,353 - INFO - Deleting VM ctest-TestBGPaasZone-35488535-99159074 2025-11-06 04:03:54,428 - INFO - Deleting VN ctest-cnz_vn-28236348 2025-11-06 04:03:54,474 - DEBUG - VN 5dfbb401-80d4-4253-9008-4d92357f50dd still in use: Unable to complete operation on network 5dfbb401-80d4-4253-9008-4d92357f50dd. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-e3ba84a9-886c-4b0e-823b-1626e2de55e0'] 2025-11-06 04:03:54,474 - WARNING - Deleting VN ctest-cnz_vn-28236348 failed..Will retry 2025-11-06 04:03:56,703 - DEBUG - Response for deleting network () 2025-11-06 04:03:57,979 - DEBUG - Skipping xmpp flap check 2025-11-06 04:03:57,980 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : PASSED[0:03:15] 2025-11-06 04:03:57,980 - INFO - -------------------------------------------------------------------------------- 2025-11-06 04:03:58,759 - INFO - Deleted project: ctest-TestBGPaasZone-35488535, ID : c1635dfb-4661-4ee7-9699-4cd3e39aa980