is in ACTIVE state now
2026-05-14 04:32:29,500 - INFO - VM name : ctest-TestBGPaasZone-94773490-99000687
2026-05-14 04:32:29,626 - DEBUG - VM ctest-TestBGPaasZone-94773490-99000687 ID is b965e8d8-749e-484e-bed1-3789ba08d7f6
2026-05-14 04:32:29,626 - DEBUG - VM ctest-TestBGPaasZone-94773490-99000687 launched on Node an-jenkins-deploy-platform-ansible-os-5780-2
2026-05-14 04:32:29,735 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/b965e8d8-749e-484e-bed1-3789ba08d7f6
2026-05-14 04:32:29,749 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/dd0a471c-a243-47ee-92a6-1b4fac9114d9
2026-05-14 04:32:30,946 - DEBUG - (True, 'PING 169.254.0.6 (169.254.0.6) 56(84) bytes of data.\r\n64 bytes from 169.254.0.6: icmp_seq=1 ttl=63 time=3.17 ms\r\n64 bytes from 169.254.0.6: icmp_seq=2 ttl=63 time=0.808 ms\r\n\r\n--- 169.254.0.6 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.808/1.988/3.168/1.180 ms')
2026-05-14 04:32:30,946 - INFO - Ping to Metadata IP 169.254.0.6 of VM ctest-TestBGPaasZone-94773490-99000687 passed
2026-05-14 04:32:31,030 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:32:31,030 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-94773490-99000687, IP 219.255.128.133, Port 22
2026-05-14 04:32:31,097 - DEBUG - Error on ssh to ubuntu@169.254.0.6:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.6/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.6/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.6/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2026-05-14 04:32:31,208 - DEBUG - VM ctest-TestBGPaasZone-94773490-99000687 is NOT ready for SSH connections, VM status: ACTIVE
2026-05-14 04:32:36,209 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:32:36,209 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-94773490-99000687, IP 219.255.128.133, Port 22
2026-05-14 04:32:36,285 - DEBUG - Error on ssh to ubuntu@169.254.0.6:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.6/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.6/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.6/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2026-05-14 04:32:36,426 - DEBUG - VM ctest-TestBGPaasZone-94773490-99000687 is NOT ready for SSH connections, VM status: ACTIVE
2026-05-14 04:32:41,427 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:32:41,427 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-94773490-99000687, IP 219.255.128.133, Port 22
2026-05-14 04:32:41,498 - DEBUG - Error on ssh to ubuntu@169.254.0.6:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.6/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.6/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.6/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2026-05-14 04:32:41,611 - DEBUG - VM ctest-TestBGPaasZone-94773490-99000687 is NOT ready for SSH connections, VM status: ACTIVE
2026-05-14 04:32:46,614 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:32:46,614 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-94773490-99000687, IP 219.255.128.133, Port 22
2026-05-14 04:32:46,790 - DEBUG - VM ctest-TestBGPaasZone-94773490-99000687 is ready for SSH connections
2026-05-14 04:32:46,947 - INFO - Created control node zone ctest-test-zone-26791636(604a46a6-a0f5-4208-85ac-d7095f19fe4c)
2026-05-14 04:32:47,363 - INFO - Created control node zone ctest-test-zone-32165397(ba86c75b-83a9-4c59-b69d-a986689dee04)
2026-05-14 04:32:47,747 - INFO - Created control node zone ctest-test-zone-06294521(ee8441a2-40e4-496f-a1bc-951f19ba6662)
2026-05-14 04:32:48,079 - INFO - Creating BGPaaS ctest-TestBGPaasZone-94773490-12329237
2026-05-14 04:32:48,167 - INFO - BGPaaS: ctest-TestBGPaasZone-94773490-12329237(50787a46-329e-4773-8812-864dc0a8d5fc)
2026-05-14 04:32:48,167 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/50787a46-329e-4773-8812-864dc0a8d5fc
2026-05-14 04:32:48,180 - INFO - verify_in_api_server passed for BGPaaS obj 50787a46-329e-4773-8812-864dc0a8d5fc
2026-05-14 04:32:48,180 - INFO - BGPaaS(50787a46-329e-4773-8812-864dc0a8d5fc): verify_on_setup passed
2026-05-14 04:32:48,180 - INFO - We will configure BGP on the VM
2026-05-14 04:32:48,675 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:32:48,675 - INFO - Configuring BGP on ctest-TestBGPaasZone-94773490-10237573
2026-05-14 04:32:48,675 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 219.255.128.131;
protocol bgp bgp_1{
local as 65000;
neighbor 219.255.128.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.131;
}
protocol bfd {
neighbor 219.255.128.129 local 219.255.128.131 multihop on;
}
EOS
, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-05-14 04:32:48,675 - DEBUG - cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 219.255.128.131;
protocol bgp bgp_1{
local as 65000;
neighbor 219.255.128.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.131;
}
protocol bfd {
neighbor 219.255.128.129 local 219.255.128.131 multihop on;
}
EOS
2026-05-14 04:32:56,588 - DEBUG - None
2026-05-14 04:32:56,588 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 219.255.128.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.131;
}
EOS
, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-05-14 04:32:56,588 - DEBUG - cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 219.255.128.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.131;
}
EOS
2026-05-14 04:32:57,205 - DEBUG - None
2026-05-14 04:32:57,205 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-05-14 04:32:57,205 - DEBUG - service bird restart
2026-05-14 04:32:59,558 - DEBUG - bird stop/waiting
bird start/running, process 1436
2026-05-14 04:32:59,558 - INFO - Attaching VMI 3d570ae9-88a3-4798-8755-d18f6a22c3fd to BGPaaS 50787a46-329e-4773-8812-864dc0a8d5fc
2026-05-14 04:32:59,655 - INFO - Attaching the VMI d889713f-fa0d-4b16-b9e8-a9c4740dfad3 to the BGPaaS 50787a46-329e-4773-8812-864dc0a8d5fc object
2026-05-14 04:32:59,655 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 219.255.128.148 dev eth0, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-05-14 04:32:59,655 - DEBUG - sudo ip addr add 219.255.128.148 dev eth0
2026-05-14 04:33:00,692 - DEBUG - None
2026-05-14 04:33:00,763 - INFO - Creating BGPaaS ctest-TestBGPaasZone-94773490-99135769
2026-05-14 04:33:00,863 - INFO - BGPaaS: ctest-TestBGPaasZone-94773490-99135769(8db7ab0b-4be2-42cf-9742-e6530c1d919f)
2026-05-14 04:33:00,863 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/8db7ab0b-4be2-42cf-9742-e6530c1d919f
2026-05-14 04:33:00,883 - INFO - verify_in_api_server passed for BGPaaS obj 8db7ab0b-4be2-42cf-9742-e6530c1d919f
2026-05-14 04:33:00,883 - INFO - BGPaaS(8db7ab0b-4be2-42cf-9742-e6530c1d919f): verify_on_setup passed
2026-05-14 04:33:00,883 - INFO - We will configure BGP on the VM
2026-05-14 04:33:01,420 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:01,420 - INFO - Configuring BGP on ctest-TestBGPaasZone-94773490-70812318
2026-05-14 04:33:01,420 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 219.255.128.132;
protocol bgp bgp_1{
local as 65000;
neighbor 219.255.128.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.132;
}
protocol bfd {
neighbor 219.255.128.129 local 219.255.128.132 multihop on;
}
EOS
, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123
2026-05-14 04:33:01,420 - DEBUG - cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 219.255.128.132;
protocol bgp bgp_1{
local as 65000;
neighbor 219.255.128.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.132;
}
protocol bfd {
neighbor 219.255.128.129 local 219.255.128.132 multihop on;
}
EOS
2026-05-14 04:33:09,085 - DEBUG - None
2026-05-14 04:33:09,085 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 219.255.128.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.132;
}
EOS
, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123
2026-05-14 04:33:09,085 - DEBUG - cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 219.255.128.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 219.255.128.132;
}
EOS
2026-05-14 04:33:09,595 - DEBUG - None
2026-05-14 04:33:09,595 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123
2026-05-14 04:33:09,595 - DEBUG - service bird restart
2026-05-14 04:33:11,434 - DEBUG - bird stop/waiting
bird start/running, process 1442
2026-05-14 04:33:11,434 - INFO - Attaching VMI 291386d6-0ccd-4c84-9cec-1af5907c6948 to BGPaaS 8db7ab0b-4be2-42cf-9742-e6530c1d919f
2026-05-14 04:33:11,519 - INFO - Attaching the VMI 83d97a1f-3080-4d24-b079-cfcf60fd83d0 to the BGPaaS 8db7ab0b-4be2-42cf-9742-e6530c1d919f object
2026-05-14 04:33:11,519 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 219.255.128.148 dev eth0, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.240, gateway password: c0ntrail123
2026-05-14 04:33:11,519 - DEBUG - sudo ip addr add 219.255.128.148 dev eth0
2026-05-14 04:33:12,178 - DEBUG - None
2026-05-14 04:33:12,178 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:12,178 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-05-14 04:33:12,195 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-05-14 04:33:12,196 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:12,215 - INFO - BGPaaS session seen in control-node 10.20.0.129
2026-05-14 04:33:12,223 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:12,224 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-05-14 04:33:12,241 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.193']
2026-05-14 04:33:18,242 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-05-14 04:33:18,256 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.193']
2026-05-14 04:33:24,257 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-05-14 04:33:24,272 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-05-14 04:33:24,273 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:24,292 - INFO - BGPaaS session seen in control-node 10.20.0.129
2026-05-14 04:33:24,301 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 219.255.128.148, host_string: ubuntu@169.254.0.6, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-05-14 04:33:24,301 - DEBUG - ping -s 56 -c 3 -W 1 219.255.128.148
2026-05-14 04:33:31,434 - DEBUG - PING 219.255.128.148 (219.255.128.148) 56(84) bytes of data.
64 bytes from 219.255.128.148: icmp_seq=1 ttl=64 time=9.43 ms
64 bytes from 219.255.128.148: icmp_seq=2 ttl=64 time=1.50 ms
64 bytes from 219.255.128.148: icmp_seq=3 ttl=64 time=1.61 ms
--- 219.255.128.148 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2005ms
rtt min/avg/max/mdev = 1.502/4.184/9.438/3.715 ms
2026-05-14 04:33:31,435 - INFO - Ping to IP 219.255.128.148 from VM ctest-TestBGPaasZone-94773490-99000687 passed
2026-05-14 04:33:31,435 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:31,435 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-05-14 04:33:31,449 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-05-14 04:33:31,449 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:31,470 - INFO - BGPaaS session seen in control-node 10.20.0.129
2026-05-14 04:33:31,477 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:31,477 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-05-14 04:33:31,491 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-05-14 04:33:31,492 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-05-14 04:33:31,514 - INFO - BGPaaS session seen in control-node 10.20.0.129
2026-05-14 04:33:31,522 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 219.255.128.148, host_string: ubuntu@169.254.0.6, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-05-14 04:33:31,522 - DEBUG - ping -s 56 -c 3 -W 1 219.255.128.148
2026-05-14 04:33:34,159 - DEBUG - PING 219.255.128.148 (219.255.128.148) 56(84) bytes of data.
64 bytes from 219.255.128.148: icmp_seq=1 ttl=64 time=6.08 ms
64 bytes from 219.255.128.148: icmp_seq=2 ttl=64 time=1.27 ms
64 bytes from 219.255.128.148: icmp_seq=3 ttl=64 time=1.33 ms
--- 219.255.128.148 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.272/2.895/6.081/2.252 ms
2026-05-14 04:33:34,159 - INFO - Ping to IP 219.255.128.148 from VM ctest-TestBGPaasZone-94773490-99000687 passed
2026-05-14 04:33:34,159 - INFO - Detaching VMI 291386d6-0ccd-4c84-9cec-1af5907c6948 from BGPaaS 8db7ab0b-4be2-42cf-9742-e6530c1d919f
2026-05-14 04:33:34,255 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-94773490-99135769(8db7ab0b-4be2-42cf-9742-e6530c1d919f)
2026-05-14 04:33:34,361 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/8db7ab0b-4be2-42cf-9742-e6530c1d919f
2026-05-14 04:33:34,463 - DEBUG - Response Code: 404
2026-05-14 04:33:34,463 - DEBUG - BGPaaS: 8db7ab0b-4be2-42cf-9742-e6530c1d919f deleted from api server
2026-05-14 04:33:34,464 - INFO - BGPaaS(8db7ab0b-4be2-42cf-9742-e6530c1d919f): verify_on_cleanup passed
2026-05-14 04:33:34,464 - INFO - Detaching VMI 3d570ae9-88a3-4798-8755-d18f6a22c3fd from BGPaaS 50787a46-329e-4773-8812-864dc0a8d5fc
2026-05-14 04:33:34,563 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-94773490-12329237(50787a46-329e-4773-8812-864dc0a8d5fc)
2026-05-14 04:33:34,660 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/50787a46-329e-4773-8812-864dc0a8d5fc
2026-05-14 04:33:34,671 - DEBUG - Response Code: 404
2026-05-14 04:33:34,671 - DEBUG - BGPaaS: 50787a46-329e-4773-8812-864dc0a8d5fc deleted from api server
2026-05-14 04:33:34,671 - INFO - BGPaaS(50787a46-329e-4773-8812-864dc0a8d5fc): verify_on_cleanup passed
2026-05-14 04:33:34,671 - INFO - Deleting Control node zone ctest-test-zone-06294521(ee8441a2-40e4-496f-a1bc-951f19ba6662)
2026-05-14 04:33:34,878 - INFO - Deleting Control node zone ctest-test-zone-32165397(ba86c75b-83a9-4c59-b69d-a986689dee04)
2026-05-14 04:33:35,058 - INFO - Deleting Control node zone ctest-test-zone-26791636(604a46a6-a0f5-4208-85ac-d7095f19fe4c)
2026-05-14 04:33:35,223 - INFO - Deleting VM ctest-TestBGPaasZone-94773490-99000687
2026-05-14 04:33:35,333 - INFO - Deleting VM ctest-TestBGPaasZone-94773490-70812318
2026-05-14 04:33:35,427 - INFO - Deleting VM ctest-TestBGPaasZone-94773490-10237573
2026-05-14 04:33:35,518 - INFO - Deleting VN ctest-cnz_vn-98792393
2026-05-14 04:33:35,577 - DEBUG - VN 331cd702-544a-4fa1-8fc1-e89925cbf68b still in use: Unable to complete operation on network 331cd702-544a-4fa1-8fc1-e89925cbf68b. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-c1aa9ec7-56dd-43ed-9f5b-003f9c10d887']
2026-05-14 04:33:35,578 - WARNING - Deleting VN ctest-cnz_vn-98792393 failed..Will retry
2026-05-14 04:33:37,835 - DEBUG - Response for deleting network ()
2026-05-14 04:33:39,125 - DEBUG - Skipping xmpp flap check
2026-05-14 04:33:39,126 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : PASSED[0:03:51]
2026-05-14 04:33:39,126 - INFO - --------------------------------------------------------------------------------
2026-05-14 04:33:40,149 - INFO - Deleted project: ctest-TestBGPaasZone-94773490, ID : f66bfa52-bdcd-41e4-bfc0-ed2220f54299