2026-01-22 03:25:05,600 - INFO - Domain Default found not creating 2026-01-22 03:25:05,772 - INFO - Project ctest-TestMd5testsOnControl-51684746 not found, creating it 2026-01-22 03:25:06,360 - INFO - Created Project:ctest-TestMd5testsOnControl-51684746, ID : 82a2ffbc-f1f3-4951-b5b9-48e646e898fb 2026-01-22 03:25:08,700 - DEBUG - Response for create_network : {'network': {'id': 'cd707189-ea26-464f-bc0b-aff05d210a37', 'name': 'test_vnv6sr', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'test_vnv6sr'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:25:08,967 - DEBUG - Response for create_subnet : {'subnet': {'id': 'f3dee9fa-0235-422c-98ae-0e49107e58d7', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': 'cd707189-ea26-464f-bc0b-aff05d210a37', 'ip_version': 6, 'cidr': '2001::101:0/120', 'allocation_pools': [{'start': '2001::101:2', 'end': '2001::101:fe'}], 'gateway_ip': '2001::101:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2001::101:2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:09,219 - DEBUG - Response for create_subnet : {'subnet': {'id': '336489eb-b9ec-4062-a407-7a66c75cc4c1', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': 'cd707189-ea26-464f-bc0b-aff05d210a37', 'ip_version': 4, 'cidr': '45.59.92.64/26', 'allocation_pools': [{'start': '45.59.92.66', 'end': '45.59.92.126'}], 'gateway_ip': '45.59.92.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '45.59.92.66', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:09,250 - INFO - Created VN test_vnv6sr 2026-01-22 03:25:09,262 - DEBUG - VN test_vnv6sr UUID is cd707189-ea26-464f-bc0b-aff05d210a37 2026-01-22 03:25:09,635 - DEBUG - Response for create_network : {'network': {'id': '98f17168-5d61-456b-a5cd-8798f50d3a11', 'name': 'test_vnv6dn', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'test_vnv6dn'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:25:09,857 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c5a98aa3-97db-46c2-97ff-f7bb125bed32', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '98f17168-5d61-456b-a5cd-8798f50d3a11', 'ip_version': 6, 'cidr': '2001::201:0/120', 'allocation_pools': [{'start': '2001::201:2', 'end': '2001::201:fe'}], 'gateway_ip': '2001::201:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2001::201:2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:10,106 - DEBUG - Response for create_subnet : {'subnet': {'id': '88f14d8a-aa27-4d97-b333-a9a997c043b9', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '98f17168-5d61-456b-a5cd-8798f50d3a11', 'ip_version': 4, 'cidr': '45.216.176.128/26', 'allocation_pools': [{'start': '45.216.176.130', 'end': '45.216.176.190'}], 'gateway_ip': '45.216.176.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '45.216.176.130', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:10,143 - INFO - Created VN test_vnv6dn 2026-01-22 03:25:10,155 - DEBUG - VN test_vnv6dn UUID is 98f17168-5d61-456b-a5cd-8798f50d3a11 2026-01-22 03:25:10,532 - DEBUG - Services list from nova: [, , , ] 2026-01-22 03:25:11,870 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-1) 2026-01-22 03:25:13,295 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-2) 2026-01-22 03:25:13,295 - INFO - Waiting for VM source_vm to be up.. 2026-01-22 03:25:13,362 - DEBUG - VM is in ERROR state now 2026-01-22 03:25:13,362 - ERROR - VM in error state. 2026-01-22 03:25:13,362 - ERROR - VM is not in ACTIVE state 2026-01-22 03:25:13,452 - ERROR - Fatal Nova Exception while getting VM detail 2026-01-22 03:25:13,452 - DEBUG - None 2026-01-22 03:25:13,452 - INFO - Waiting for VM dest_vm to be up.. 2026-01-22 03:25:13,510 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-01-22 03:25:18,594 - DEBUG - VM is in ACTIVE state now 2026-01-22 03:25:18,594 - INFO - VM name : dest_vm 2026-01-22 03:25:18,680 - DEBUG - VM dest_vm ID is 24e2d824-f5d6-42bc-a2b6-4ea976867f20 2026-01-22 03:25:18,705 - DEBUG - VM dest_vm launched on Node an-jenkins-deploy-platform-ansible-os-5055-2 2026-01-22 03:25:18,802 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/24e2d824-f5d6-42bc-a2b6-4ea976867f20 2026-01-22 03:25:19,113 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/24e2d824-f5d6-42bc-a2b6-4ea976867f20 2026-01-22 03:25:19,158 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/493ee89e-636f-4fb9-83c7-b866ba435204 2026-01-22 03:25:22,309 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1025ms') 2026-01-22 03:25:22,310 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM dest_vm failed! 2026-01-22 03:25:22,325 - DEBUG - Gateway for vn default-domain:ctest-TestMd5testsOnControl-51684746:test_vnv6dn is 2001::201:1 and allocation pool is NOT set 2026-01-22 03:25:22,325 - DEBUG - Gateway for vn default-domain:ctest-TestMd5testsOnControl-51684746:test_vnv6dn is 45.216.176.129 and allocation pool is NOT set 2026-01-22 03:25:24,349 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=8.86 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.87 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.873/5.868/8.863/2.995 ms') 2026-01-22 03:25:24,349 - INFO - Ping to Metadata IP 169.254.0.3 of VM dest_vm passed 2026-01-22 03:25:24,420 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-01-22 03:25:24,420 - DEBUG - Waiting to SSH to VM dest_vm, IP 45.216.176.131, Port 22 2026-01-22 03:25:24,436 - DEBUG - Error on ssh to cirros@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-01-22 03:25:24,520 - DEBUG - VM dest_vm is NOT ready for SSH connections, VM status: ACTIVE 2026-01-22 03:25:29,521 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-01-22 03:25:29,521 - DEBUG - Waiting to SSH to VM dest_vm, IP 45.216.176.131, Port 22 2026-01-22 03:25:29,594 - DEBUG - VM dest_vm is ready for SSH connections 2026-01-22 03:25:29,601 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:test_vnv6sr, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = 0, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:test_vnv6dn, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = 0, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2026-01-22 03:25:29,934 - INFO - Attaching policy allow_all to vn test_vnv6sr 2026-01-22 03:25:30,322 - INFO - Attaching policy allow_all to vn test_vnv6dn 2026-01-22 03:25:30,573 - INFO - Configure the policy with allow any 2026-01-22 03:25:30,962 - DEBUG - Response for create_network : {'network': {'id': '1a8d71c5-f00b-4a6a-af59-106e03a8e024', 'name': 'vn1', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'vn1'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:25:31,217 - DEBUG - Response for create_subnet : {'subnet': {'id': 'cee61a19-7dfe-4782-ba27-72f3e05f68b0', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '1a8d71c5-f00b-4a6a-af59-106e03a8e024', 'ip_version': 4, 'cidr': '10.1.1.0/24', 'allocation_pools': [{'start': '10.1.1.2', 'end': '10.1.1.254'}], 'gateway_ip': '10.1.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '10.1.1.2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:31,245 - INFO - Created VN vn1 2026-01-22 03:25:31,258 - DEBUG - VN vn1 UUID is 1a8d71c5-f00b-4a6a-af59-106e03a8e024 2026-01-22 03:25:31,610 - DEBUG - Response for create_network : {'network': {'id': '2694ea04-617f-4dcf-bc98-9f2625736f7a', 'name': 'vn2', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'vn2'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:25:31,805 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c63e59a0-74e6-447c-b857-8e868cbe9da0', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '2694ea04-617f-4dcf-bc98-9f2625736f7a', 'ip_version': 4, 'cidr': '20.1.1.0/24', 'allocation_pools': [{'start': '20.1.1.2', 'end': '20.1.1.254'}], 'gateway_ip': '20.1.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '20.1.1.2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:31,827 - INFO - Created VN vn2 2026-01-22 03:25:31,838 - DEBUG - VN vn2 UUID is 2694ea04-617f-4dcf-bc98-9f2625736f7a 2026-01-22 03:25:31,916 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:vn1, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = 0, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:vn2, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = 0, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2026-01-22 03:25:32,253 - INFO - Attaching policy sec_grp_policy_ctest-random-02636350 to vn vn1 2026-01-22 03:25:32,619 - INFO - Attaching policy sec_grp_policy_ctest-random-02636350 to vn vn2 2026-01-22 03:25:34,382 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-1) 2026-01-22 03:25:35,892 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-2) 2026-01-22 03:25:35,954 - INFO - ================================================================================ 2026-01-22 03:25:35,954 - INFO - STARTING TEST : test_check_per_peer_on_control 2026-01-22 03:25:35,954 - INFO - TEST DESCRIPTION : Description: Verify per peer md5 and specific protocol on all ports and policy with allow all between VN's 2026-01-22 03:25:36,630 - DEBUG - Skipping xmpp flap check 2026-01-22 03:25:36,631 - INFO - Initial checks done. Running the testcase now 2026-01-22 03:25:36,631 - INFO - 2026-01-22 03:25:36,631 - INFO - config per peer with auth_data None 2026-01-22 03:25:37,239 - INFO - configure MD5 on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:25:37,323 - INFO - config per peer with auth_data None 2026-01-22 03:25:37,899 - INFO - configure MD5 on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:25:37,995 - INFO - config per peer with auth_data None 2026-01-22 03:25:38,555 - INFO - configure MD5 on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:25:38,636 - INFO - check BGP peering is up before setting md5 config 2026-01-22 03:25:38,636 - INFO - Check BGP staus between peers 2026-01-22 03:25:38,659 - INFO - config per peer with auth_data {'key_items': [{'key': 'juniper', 'key_id': 0}], 'key_type': 'md5'} 2026-01-22 03:25:39,255 - INFO - Check BGP staus between peers 2026-01-22 03:25:39,269 - DEBUG - Check the BGP connection on 10.0.0.38 2026-01-22 03:25:39,269 - DEBUG - Check the BGP connection on 10.0.0.38 2026-01-22 03:25:49,270 - INFO - Check BGP staus between peers 2026-01-22 03:25:49,283 - INFO - remove md5 configured on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:25:49,283 - INFO - config per peer with auth_data None 2026-01-22 03:25:49,807 - INFO - configure MD5 on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:25:49,891 - INFO - remove md5 configured on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:25:49,891 - INFO - config per peer with auth_data None 2026-01-22 03:25:50,511 - INFO - configure MD5 on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:25:50,595 - INFO - remove md5 configured on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:25:50,595 - INFO - config per peer with auth_data None 2026-01-22 03:25:51,179 - INFO - configure MD5 on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:25:51,275 - INFO - Skip interface_detach for VM vn2_vm_0 in state BUILD 2026-01-22 03:25:51,275 - INFO - Deleting VM vn2_vm_0 2026-01-22 03:25:51,275 - ERROR - VM vn2_vm_0 has failed to come up 2026-01-22 03:25:51,275 - ERROR - Nova failed to get host of the VM 2026-01-22 03:25:51,275 - ERROR - VM vn2_vm_0 has failed to come up 2026-01-22 03:25:51,275 - ERROR - Nova failed to get host of the VM 2026-01-22 03:25:51,341 - INFO - Skip interface_detach for VM vn1_vm_0 in state ERROR 2026-01-22 03:25:51,341 - INFO - Deleting VM vn1_vm_0 2026-01-22 03:25:51,341 - ERROR - VM vn1_vm_0 has failed to come up 2026-01-22 03:25:51,341 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2026-01-22T03:25:34Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2026-01-22 03:25:51,343 - INFO - Dettaching policy sec_grp_policy_ctest-random-02636350 from vn vn2 2026-01-22 03:25:51,509 - INFO - Dettaching policy sec_grp_policy_ctest-random-02636350 from vn vn1 2026-01-22 03:25:51,742 - INFO - Deleted policy sec_grp_policy_ctest-random-02636350 2026-01-22 03:25:51,742 - INFO - Deleting VN vn2 2026-01-22 03:25:51,887 - DEBUG - VN 2694ea04-617f-4dcf-bc98-9f2625736f7a still in use: Unable to complete operation on network 2694ea04-617f-4dcf-bc98-9f2625736f7a. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-c0e880b7-6c34-4277-99bf-25802d9c377d'] 2026-01-22 03:25:51,888 - WARNING - Deleting VN vn2 failed..Will retry 2026-01-22 03:25:54,060 - DEBUG - Response for deleting network () 2026-01-22 03:25:54,060 - INFO - Deleting VN vn1 2026-01-22 03:25:54,222 - DEBUG - Response for deleting network () 2026-01-22 03:25:54,223 - INFO - Dettaching policy allow_all from vn test_vnv6dn 2026-01-22 03:25:54,397 - INFO - Dettaching policy allow_all from vn test_vnv6sr 2026-01-22 03:25:54,655 - INFO - Deleted policy allow_all 2026-01-22 03:25:54,656 - INFO - Deleting VM dest_vm 2026-01-22 03:25:54,741 - INFO - Skip interface_detach for VM source_vm in state ERROR 2026-01-22 03:25:54,741 - INFO - Deleting VM source_vm 2026-01-22 03:25:54,741 - ERROR - VM source_vm has failed to come up 2026-01-22 03:25:54,741 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2026-01-22T03:25:11Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2026-01-22 03:25:54,741 - INFO - Deleting VN test_vnv6dn 2026-01-22 03:25:54,784 - DEBUG - VN 98f17168-5d61-456b-a5cd-8798f50d3a11 still in use: Unable to complete operation on network 98f17168-5d61-456b-a5cd-8798f50d3a11. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-d96859cd-e858-4663-8d09-9931e34e3e29'] 2026-01-22 03:25:54,784 - WARNING - Deleting VN test_vnv6dn failed..Will retry 2026-01-22 03:25:56,964 - DEBUG - Response for deleting network () 2026-01-22 03:25:56,964 - INFO - Deleting VN test_vnv6sr 2026-01-22 03:25:57,120 - DEBUG - Response for deleting network () 2026-01-22 03:25:57,754 - DEBUG - Skipping xmpp flap check
2026-01-22 03:25:57,754 - INFO - END TEST : test_check_per_peer_on_control : PASSED[0:00:22]
2026-01-22 03:25:57,754 - INFO - -------------------------------------------------------------------------------- 2026-01-22 03:25:58,017 - DEBUG - Response for create_network : {'network': {'id': '72fdf443-d0b7-4aaa-8083-f44e9f469f86', 'name': 'test_vnv6sr', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'test_vnv6sr'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:25:58,253 - DEBUG - Response for create_subnet : {'subnet': {'id': 'bb7985f9-18bd-40f6-8df0-d1e911041470', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '72fdf443-d0b7-4aaa-8083-f44e9f469f86', 'ip_version': 6, 'cidr': '2001::101:0/120', 'allocation_pools': [{'start': '2001::101:2', 'end': '2001::101:fe'}], 'gateway_ip': '2001::101:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2001::101:2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:58,485 - DEBUG - Response for create_subnet : {'subnet': {'id': '67c67a0a-c728-44f5-95f1-17f9186aea89', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '72fdf443-d0b7-4aaa-8083-f44e9f469f86', 'ip_version': 4, 'cidr': '199.0.71.64/26', 'allocation_pools': [{'start': '199.0.71.66', 'end': '199.0.71.126'}], 'gateway_ip': '199.0.71.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '199.0.71.66', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:58,510 - INFO - Created VN test_vnv6sr 2026-01-22 03:25:58,523 - DEBUG - VN test_vnv6sr UUID is 72fdf443-d0b7-4aaa-8083-f44e9f469f86 2026-01-22 03:25:58,926 - DEBUG - Response for create_network : {'network': {'id': 'a60ef9ef-8b19-4408-9f14-d85693fb6c2f', 'name': 'test_vnv6dn', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'test_vnv6dn'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:25:59,174 - DEBUG - Response for create_subnet : {'subnet': {'id': 'efd8b8e1-490e-4e38-88f1-229344c73683', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': 'a60ef9ef-8b19-4408-9f14-d85693fb6c2f', 'ip_version': 6, 'cidr': '2001::201:0/120', 'allocation_pools': [{'start': '2001::201:2', 'end': '2001::201:fe'}], 'gateway_ip': '2001::201:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2001::201:2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:59,442 - DEBUG - Response for create_subnet : {'subnet': {'id': '04142ac0-1e6d-4074-9027-1faa32d2b2c0', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': 'a60ef9ef-8b19-4408-9f14-d85693fb6c2f', 'ip_version': 4, 'cidr': '68.191.82.0/26', 'allocation_pools': [{'start': '68.191.82.2', 'end': '68.191.82.62'}], 'gateway_ip': '68.191.82.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '68.191.82.2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:25:59,467 - INFO - Created VN test_vnv6dn 2026-01-22 03:25:59,489 - DEBUG - VN test_vnv6dn UUID is a60ef9ef-8b19-4408-9f14-d85693fb6c2f 2026-01-22 03:26:00,063 - DEBUG - VM source_vm already present, not creating it 2026-01-22 03:26:01,613 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-1) 2026-01-22 03:26:01,613 - INFO - Waiting for VM source_vm to be up.. 2026-01-22 03:26:01,685 - DEBUG - VM is in ERROR state now 2026-01-22 03:26:01,685 - ERROR - VM in error state. 2026-01-22 03:26:01,685 - ERROR - VM is not in ACTIVE state 2026-01-22 03:26:01,791 - ERROR - Fatal Nova Exception while getting VM detail 2026-01-22 03:26:01,791 - DEBUG - None 2026-01-22 03:26:01,791 - INFO - Waiting for VM dest_vm to be up.. 2026-01-22 03:26:01,869 - DEBUG - VM is in ERROR state now 2026-01-22 03:26:01,869 - ERROR - VM in error state. 2026-01-22 03:26:01,869 - ERROR - VM is not in ACTIVE state 2026-01-22 03:26:01,971 - ERROR - Fatal Nova Exception while getting VM detail 2026-01-22 03:26:01,971 - DEBUG - None 2026-01-22 03:26:01,978 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:test_vnv6sr, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = 0, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:test_vnv6dn, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = 0, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2026-01-22 03:26:02,331 - INFO - Attaching policy allow_all to vn test_vnv6sr 2026-01-22 03:26:02,732 - INFO - Attaching policy allow_all to vn test_vnv6dn 2026-01-22 03:26:03,024 - INFO - Configure the policy with allow any 2026-01-22 03:26:03,381 - DEBUG - Response for create_network : {'network': {'id': '28a4b519-1f17-4731-8169-16cbdd22fe7b', 'name': 'vn1', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'vn1'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:26:03,755 - DEBUG - Response for create_subnet : {'subnet': {'id': '580384b2-5dd9-49f0-83ca-35b1cdd7ae6d', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '28a4b519-1f17-4731-8169-16cbdd22fe7b', 'ip_version': 4, 'cidr': '10.1.1.0/24', 'allocation_pools': [{'start': '10.1.1.2', 'end': '10.1.1.254'}], 'gateway_ip': '10.1.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '10.1.1.2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:26:03,786 - INFO - Created VN vn1 2026-01-22 03:26:03,812 - DEBUG - VN vn1 UUID is 28a4b519-1f17-4731-8169-16cbdd22fe7b 2026-01-22 03:26:04,231 - DEBUG - Response for create_network : {'network': {'id': '78080fb9-56b9-4daa-abb7-f2e8351185f8', 'name': 'vn2', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'project_id': '82a2ffbcf1f34951b5b948e646e898fb', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMd5testsOnControl-51684746', 'vn2'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:26:04,496 - DEBUG - Response for create_subnet : {'subnet': {'id': '35f50688-25fb-4a1c-b958-d96fb02ea398', 'name': '', 'tenant_id': '82a2ffbcf1f34951b5b948e646e898fb', 'network_id': '78080fb9-56b9-4daa-abb7-f2e8351185f8', 'ip_version': 4, 'cidr': '20.1.1.0/24', 'allocation_pools': [{'start': '20.1.1.2', 'end': '20.1.1.254'}], 'gateway_ip': '20.1.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '20.1.1.2', 'tags': [], 'project_id': '82a2ffbcf1f34951b5b948e646e898fb'}} 2026-01-22 03:26:04,517 - INFO - Created VN vn2 2026-01-22 03:26:04,528 - DEBUG - VN vn2 UUID is 78080fb9-56b9-4daa-abb7-f2e8351185f8 2026-01-22 03:26:04,607 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:vn1, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = 0, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestMd5testsOnControl-51684746:vn2, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = 0, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2026-01-22 03:26:05,054 - INFO - Attaching policy sec_grp_policy_ctest-random-51437875 to vn vn1 2026-01-22 03:26:05,433 - INFO - Attaching policy sec_grp_policy_ctest-random-51437875 to vn vn2 2026-01-22 03:26:06,220 - DEBUG - VM vn1_vm_0 already present, not creating it 2026-01-22 03:26:07,763 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-2) 2026-01-22 03:26:07,827 - INFO - ================================================================================ 2026-01-22 03:26:07,827 - INFO - STARTING TEST : test_create_md5_on_control 2026-01-22 03:26:07,827 - INFO - TEST DESCRIPTION : Description: Verify md5 with allow specific protocol on all ports and policy with allow all between VN's 2026-01-22 03:26:08,505 - DEBUG - Skipping xmpp flap check 2026-01-22 03:26:08,505 - INFO - Initial checks done. Running the testcase now 2026-01-22 03:26:08,506 - INFO - 2026-01-22 03:26:08,506 - INFO - config md5 on each host with auth_data None 2026-01-22 03:26:08,506 - INFO - config per peer with auth_data None 2026-01-22 03:26:09,099 - INFO - configure MD5 on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:26:09,178 - INFO - config per peer with auth_data None 2026-01-22 03:26:09,771 - INFO - configure MD5 on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:26:09,855 - INFO - config per peer with auth_data None 2026-01-22 03:26:10,447 - INFO - configure MD5 on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:26:10,535 - INFO - check if BGP between peers are up before setting md5 2026-01-22 03:26:10,535 - INFO - Check BGP staus between peers 2026-01-22 03:26:10,545 - INFO - setting MD5 config on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:26:10,545 - INFO - configure MD5 on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:26:10,631 - INFO - setting MD5 config on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:26:10,631 - INFO - configure MD5 on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:26:10,707 - INFO - setting MD5 config on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:26:10,707 - INFO - configure MD5 on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:26:10,779 - INFO - check if BGP peering is up after md5 config 2026-01-22 03:26:10,779 - INFO - Check BGP staus between peers 2026-01-22 03:26:10,789 - DEBUG - Check the BGP connection on 10.0.0.38 2026-01-22 03:26:10,789 - DEBUG - Check the BGP connection on 10.0.0.38 2026-01-22 03:26:20,789 - INFO - Check BGP staus between peers 2026-01-22 03:26:20,800 - INFO - remove md5 configured on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:26:20,800 - INFO - config per peer with auth_data None 2026-01-22 03:26:21,335 - INFO - configure MD5 on host aca0c890-b03f-45f7-8033-20b1ad20bfa7 2026-01-22 03:26:21,427 - INFO - remove md5 configured on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:26:21,427 - INFO - config per peer with auth_data None 2026-01-22 03:26:21,974 - INFO - configure MD5 on host e01006e1-58dc-478a-9c9a-6dce4e433be1 2026-01-22 03:26:22,055 - INFO - remove md5 configured on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:26:22,055 - INFO - config per peer with auth_data None 2026-01-22 03:26:22,615 - INFO - configure MD5 on host 95000387-5ece-41ef-948f-dcf893689f99 2026-01-22 03:26:22,695 - INFO - Skip interface_detach for VM vn2_vm_0 in state BUILD 2026-01-22 03:26:22,695 - INFO - Deleting VM vn2_vm_0 2026-01-22 03:26:22,695 - ERROR - VM vn2_vm_0 has failed to come up 2026-01-22 03:26:22,695 - ERROR - Nova failed to get host of the VM 2026-01-22 03:26:22,695 - ERROR - VM vn2_vm_0 has failed to come up 2026-01-22 03:26:22,695 - ERROR - Nova failed to get host of the VM 2026-01-22 03:26:22,764 - INFO - Skipping the deletion of VM vn1_vm_0 2026-01-22 03:26:22,765 - INFO - Dettaching policy sec_grp_policy_ctest-random-51437875 from vn vn2 2026-01-22 03:26:22,922 - INFO - Dettaching policy sec_grp_policy_ctest-random-51437875 from vn vn1 2026-01-22 03:26:23,166 - INFO - Deleted policy sec_grp_policy_ctest-random-51437875 2026-01-22 03:26:23,166 - INFO - Deleting VN vn2 2026-01-22 03:26:23,205 - DEBUG - VN 78080fb9-56b9-4daa-abb7-f2e8351185f8 still in use: Unable to complete operation on network 78080fb9-56b9-4daa-abb7-f2e8351185f8. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-9aea97cd-4efe-4398-a8a0-eb3818f22e21'] 2026-01-22 03:26:23,205 - WARNING - Deleting VN vn2 failed..Will retry 2026-01-22 03:26:25,353 - DEBUG - Response for deleting network () 2026-01-22 03:26:25,354 - INFO - Deleting VN vn1 2026-01-22 03:26:25,527 - DEBUG - Response for deleting network () 2026-01-22 03:26:25,527 - INFO - Dettaching policy allow_all from vn test_vnv6dn 2026-01-22 03:26:25,705 - INFO - Dettaching policy allow_all from vn test_vnv6sr 2026-01-22 03:26:25,961 - INFO - Deleted policy allow_all 2026-01-22 03:26:25,961 - INFO - Skip interface_detach for VM dest_vm in state ERROR 2026-01-22 03:26:25,961 - INFO - Deleting VM dest_vm 2026-01-22 03:26:25,961 - ERROR - VM dest_vm has failed to come up 2026-01-22 03:26:25,962 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2026-01-22T03:26:01Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2026-01-22 03:26:25,962 - INFO - Skipping the deletion of VM source_vm 2026-01-22 03:26:25,962 - INFO - Deleting VN test_vnv6dn 2026-01-22 03:26:26,125 - DEBUG - Response for deleting network () 2026-01-22 03:26:26,125 - INFO - Deleting VN test_vnv6sr 2026-01-22 03:26:26,330 - DEBUG - Response for deleting network () 2026-01-22 03:26:26,970 - DEBUG - Skipping xmpp flap check
2026-01-22 03:26:26,970 - INFO - END TEST : test_create_md5_on_control : PASSED[0:00:19]
2026-01-22 03:26:26,970 - INFO - -------------------------------------------------------------------------------- 2026-01-22 03:26:27,905 - INFO - Deleted project: ctest-TestMd5testsOnControl-51684746, ID : 82a2ffbc-f1f3-4951-b5b9-48e646e898fb