2026-01-22 03:29:15,605 - INFO - Domain Default found not creating 2026-01-22 03:29:15,773 - INFO - Project ctest-TestBasicVMVN0-95493504 not found, creating it 2026-01-22 03:29:16,369 - INFO - Created Project:ctest-TestBasicVMVN0-95493504, ID : f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:17,969 - INFO - ================================================================================ 2026-01-22 03:29:17,969 - INFO - STARTING TEST : test_control_node_switchover 2026-01-22 03:29:17,969 - INFO - TEST DESCRIPTION : Stop the control node and check peering with agent fallback to other control node. 1. Pick one VN from respource pool which has 2 VM's in it 2. Verify ping between VM's 3. Find active control node in cluster by agent inspect 4. Stop control service on active control node 5. Verify agents are connected to new active control-node using xmpp connections 6. Bring back control service on previous active node 7. Verify ping between VM's again after bringing up control serveice Pass criteria: Step 2,5 and 7 should pass 2026-01-22 03:29:18,612 - DEBUG - Skipping xmpp flap check 2026-01-22 03:29:18,612 - INFO - Initial checks done. Running the testcase now 2026-01-22 03:29:18,612 - INFO - 2026-01-22 03:29:19,361 - DEBUG - Response for create_network : {'network': {'id': '96965a3e-bb9a-45cb-a1aa-5c809c200d54', 'name': 'ctest-vn1-88217239', 'tenant_id': 'f95967f510b0498ab93201ed7d206c1e', 'project_id': 'f95967f510b0498ab93201ed7d206c1e', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBasicVMVN0-95493504', 'ctest-vn1-88217239'], 'port_security_enabled': True, 'description': ''}} 2026-01-22 03:29:19,683 - DEBUG - Response for create_subnet : {'subnet': {'id': '3f4d940d-7b62-4eda-90c7-c009a3fdd784', 'name': '', 'tenant_id': 'f95967f510b0498ab93201ed7d206c1e', 'network_id': '96965a3e-bb9a-45cb-a1aa-5c809c200d54', 'ip_version': 4, 'cidr': '192.168.1.0/24', 'allocation_pools': [{'start': '192.168.1.2', 'end': '192.168.1.254'}], 'gateway_ip': '192.168.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '192.168.1.2', 'tags': [], 'project_id': 'f95967f510b0498ab93201ed7d206c1e'}} 2026-01-22 03:29:19,711 - INFO - Created VN ctest-vn1-88217239 2026-01-22 03:29:19,722 - DEBUG - VN ctest-vn1-88217239 UUID is 96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:19,805 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:20,121 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:20,160 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:20,179 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:20,287 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:20,299 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:20,315 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/6673168f-c47d-4332-b3d7-c5a745ce0b8b 2026-01-22 03:29:20,331 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/6673168f-c47d-4332-b3d7-c5a745ce0b8b 2026-01-22 03:29:20,339 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/83ba8793-66c2-441d-b2d5-c87f3f390770 2026-01-22 03:29:20,346 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:20,346 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:20,356 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/6673168f-c47d-4332-b3d7-c5a745ce0b8b 2026-01-22 03:29:20,375 - INFO - Verified VN network id 14 for VN 96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:20,375 - INFO - Verifications in API Server for VN ctest-vn1-88217239 passed 2026-01-22 03:29:20,375 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:20,384 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/6673168f-c47d-4332-b3d7-c5a745ce0b8b 2026-01-22 03:29:20,392 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/6673168f-c47d-4332-b3d7-c5a745ce0b8b 2026-01-22 03:29:20,400 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/83ba8793-66c2-441d-b2d5-c87f3f390770 2026-01-22 03:29:20,412 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10850959577677710795', 'uuid-lslong': '11649225093592124756'}, 'enable': 'true', 'created': '2026-01-22T03:29:19', 'last-modified': '2026-01-22T03:29:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.784073'} 2026-01-22 03:29:20,419 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:20,425 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10850959577677710795', 'uuid-lslong': '11649225093592124756'}, 'enable': 'true', 'created': '2026-01-22T03:29:19', 'last-modified': '2026-01-22T03:29:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.792023'} 2026-01-22 03:29:20,432 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:20,437 - DEBUG - Control-node 10.0.0.131 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10850959577677710795', 'uuid-lslong': '11649225093592124756'}, 'enable': 'true', 'created': '2026-01-22T03:29:19', 'last-modified': '2026-01-22T03:29:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.808788'} 2026-01-22 03:29:20,443 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:20,443 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-88217239 passed 2026-01-22 03:29:20,443 - DEBUG - ====Verifying policy data for ctest-vn1-88217239 in API_Server ====== 2026-01-22 03:29:20,443 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:20,450 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:20,464 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:20,558 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/96965a3e-bb9a-45cb-a1aa-5c809c200d54 2026-01-22 03:29:20,568 - DEBUG - =>VN ctest-vn1-88217239 has no policy to be verified 2026-01-22 03:29:20,568 - DEBUG - Verifying the vn in opserver 2026-01-22 03:29:20,569 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239 virtual network link through opserver 10.0.0.38 2026-01-22 03:29:20,569 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2026-01-22 03:29:20,875 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2026-01-22 03:29:20,946 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239?flat'} 2026-01-22 03:29:20,946 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239 virtual network link through opserver 10.0.0.50 2026-01-22 03:29:20,946 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2026-01-22 03:29:21,254 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2026-01-22 03:29:21,330 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239?flat'} 2026-01-22 03:29:21,330 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239 virtual network link through opserver 10.0.0.131 2026-01-22 03:29:21,330 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-networks 2026-01-22 03:29:21,640 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-networks 2026-01-22 03:29:21,706 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239', 'href': 'http://10.0.0.131:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239?flat'} 2026-01-22 03:29:21,706 - INFO - Validated that VN default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-88217239 is found in opserver 2026-01-22 03:29:21,729 - DEBUG - Do not have enough data to verify VN in agent 2026-01-22 03:29:21,740 - DEBUG - VRF ids for VN ctest-vn1-88217239: {} 2026-01-22 03:29:22,000 - DEBUG - Services list from nova: [, , , ] 2026-01-22 03:29:23,327 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-1) 2026-01-22 03:29:24,743 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-2) 2026-01-22 03:29:24,743 - INFO - Waiting for VM ctest-vn1_vm1-81465951 to be up.. 2026-01-22 03:29:24,806 - DEBUG - VM is in ERROR state now 2026-01-22 03:29:24,806 - ERROR - VM in error state. 2026-01-22 03:29:24,806 - ERROR - VM is not in ACTIVE state 2026-01-22 03:29:24,884 - ERROR - Fatal Nova Exception while getting VM detail 2026-01-22 03:29:24,884 - DEBUG - None 2026-01-22 03:29:24,887 - INFO - Skip interface_detach for VM ctest-vn1_vm2-65907821 in state BUILD 2026-01-22 03:29:24,887 - INFO - Deleting VM ctest-vn1_vm2-65907821 2026-01-22 03:29:24,914 - ERROR - VM ctest-vn1_vm2-65907821 has failed to come up 2026-01-22 03:29:24,918 - ERROR - Nova failed to get host of the VM 2026-01-22 03:29:24,918 - ERROR - VM ctest-vn1_vm2-65907821 has failed to come up 2026-01-22 03:29:24,918 - ERROR - Nova failed to get host of the VM 2026-01-22 03:29:24,995 - INFO - Skip interface_detach for VM ctest-vn1_vm1-81465951 in state ERROR 2026-01-22 03:29:24,995 - INFO - Deleting VM ctest-vn1_vm1-81465951 2026-01-22 03:29:24,995 - ERROR - VM ctest-vn1_vm1-81465951 has failed to come up 2026-01-22 03:29:24,995 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2026-01-22T03:29:23Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2026-01-22 03:29:24,995 - INFO - Deleting VN ctest-vn1-88217239 2026-01-22 03:29:25,176 - DEBUG - Response for deleting network () 2026-01-22 03:29:25,177 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/6673168f-c47d-4332-b3d7-c5a745ce0b8b 2026-01-22 03:29:25,182 - DEBUG - Response Code: 404 2026-01-22 03:29:25,182 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:25,188 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:25,203 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:25,301 - INFO - Validated that VN ctest-vn1-88217239 is not found in API Server 2026-01-22 03:29:25,316 - DEBUG - VN ctest-vn1-88217239 is not present in Agent 10.0.0.23 2026-01-22 03:29:25,326 - DEBUG - VN ctest-vn1-88217239 is not present in Agent 10.0.0.141 2026-01-22 03:29:25,326 - INFO - Validated that VN ctest-vn1-88217239 is not in any agent 2026-01-22 03:29:25,356 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn1-88217239 info 2026-01-22 03:29:26,012 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Thu Jan 22 03:29:24 2026 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_control_node_switchover(self=) 635 vm1_fixture = self.create_vm(vn1_fixture, vn1_vm1_name) 636 vm2_fixture = self.create_vm(vn1_fixture, vn1_vm2_name) 637 assert vm1_fixture.wait_till_vm_is_up() 638 assert vm2_fixture.wait_till_vm_is_up() 639 assert vm1_fixture.ping_to_ip(vm2_fixture.vm_ip) vm1_fixture = vm1_fixture.wait_till_vm_is_up = > AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 637, in test_control_node_switchover assert vm1_fixture.wait_till_vm_is_up() AssertionError 2026-01-22 03:29:26,013 - DEBUG - Skipping xmpp flap check 2026-01-22 03:29:26,013 - INFO -
2026-01-22 03:29:26,013 - INFO - END TEST : test_control_node_switchover : FAILED[0:00:09]
2026-01-22 03:29:26,013 - INFO - -------------------------------------------------------------------------------- 2026-01-22 03:29:26,016 - INFO - ================================================================================ 2026-01-22 03:29:26,016 - INFO - STARTING TEST : test_process_restart_in_policy_between_vns 2026-01-22 03:29:26,016 - INFO - TEST DESCRIPTION : Test to validate that with policy having rule to check icmp fwding between VMs on different VNs , ping between VMs should pass with process restarts 1. Pick 2 VN's from resource pool which has one VM each 2. Create policy with icmp allow rule between those VN's and bind it networks 3. Ping from one VM to another VM 4. Restart process 'vrouter' and 'control' on setup 5. Ping again between VM's after process restart Pass criteria: Step 2,3,4 and 5 should pass 2026-01-22 03:29:26,680 - DEBUG - Skipping xmpp flap check 2026-01-22 03:29:26,680 - INFO - Initial checks done. Running the testcase now 2026-01-22 03:29:26,680 - INFO - 2026-01-22 03:29:26,686 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2026-01-22 03:29:26,955 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2026-01-22 03:29:27,491 - INFO - Created VN ctest-vn1-11629845, UUID :9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:27,819 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:27,825 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:27,836 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:27,939 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:27,951 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:27,960 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/ef356bcd-6022-41b9-9e4d-d1a738dc425e 2026-01-22 03:29:27,967 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/ef356bcd-6022-41b9-9e4d-d1a738dc425e 2026-01-22 03:29:27,976 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/2e635539-f060-4477-8daa-2312307cef3c 2026-01-22 03:29:27,983 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:27,983 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:27,991 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/ef356bcd-6022-41b9-9e4d-d1a738dc425e 2026-01-22 03:29:28,008 - INFO - Verified VN network id 14 for VN 9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:28,009 - INFO - Verifications in API Server for VN ctest-vn1-11629845 passed 2026-01-22 03:29:28,009 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:28,018 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/ef356bcd-6022-41b9-9e4d-d1a738dc425e 2026-01-22 03:29:28,025 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/ef356bcd-6022-41b9-9e4d-d1a738dc425e 2026-01-22 03:29:28,033 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/2e635539-f060-4477-8daa-2312307cef3c 2026-01-22 03:29:28,043 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11312778582624387294', 'uuid-lslong': '13033779655064284665'}, 'enable': 'true', 'created': '2026-01-22T03:29:27', 'last-modified': '2026-01-22T03:29:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.409098'} 2026-01-22 03:29:28,048 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:28,053 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11312778582624387294', 'uuid-lslong': '13033779655064284665'}, 'enable': 'true', 'created': '2026-01-22T03:29:27', 'last-modified': '2026-01-22T03:29:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.420243'} 2026-01-22 03:29:28,059 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:28,065 - DEBUG - Control-node 10.0.0.131 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11312778582624387294', 'uuid-lslong': '13033779655064284665'}, 'enable': 'true', 'created': '2026-01-22T03:29:27', 'last-modified': '2026-01-22T03:29:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.436066'} 2026-01-22 03:29:28,070 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-01-22 03:29:28,071 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-11629845 passed 2026-01-22 03:29:28,071 - DEBUG - ====Verifying policy data for ctest-vn1-11629845 in API_Server ====== 2026-01-22 03:29:28,071 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:28,077 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:28,090 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:28,200 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/9cff102e-e7f9-40de-b4e1-498a54d1edf9 2026-01-22 03:29:28,209 - DEBUG - =>VN ctest-vn1-11629845 has no policy to be verified 2026-01-22 03:29:28,209 - DEBUG - Verifying the vn in opserver 2026-01-22 03:29:28,209 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845 virtual network link through opserver 10.0.0.38 2026-01-22 03:29:28,209 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2026-01-22 03:29:28,275 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845?flat'} 2026-01-22 03:29:28,275 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845 virtual network link through opserver 10.0.0.50 2026-01-22 03:29:28,275 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2026-01-22 03:29:28,355 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845?flat'} 2026-01-22 03:29:28,355 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845 virtual network link through opserver 10.0.0.131 2026-01-22 03:29:28,355 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-networks 2026-01-22 03:29:28,420 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845', 'href': 'http://10.0.0.131:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845?flat'} 2026-01-22 03:29:28,420 - INFO - Validated that VN default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn1-11629845 is found in opserver 2026-01-22 03:29:28,444 - DEBUG - Do not have enough data to verify VN in agent 2026-01-22 03:29:28,455 - DEBUG - VRF ids for VN ctest-vn1-11629845: {} 2026-01-22 03:29:29,127 - INFO - Created VN ctest-vn2-58807010, UUID :0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,471 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:29,478 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:29,489 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:29,589 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,601 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,613 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/b80b2685-4c49-4beb-87fa-1aa488869f20 2026-01-22 03:29:29,623 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/b80b2685-4c49-4beb-87fa-1aa488869f20 2026-01-22 03:29:29,633 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/3d5cf819-d036-4697-a61f-d0aa32d09c1f 2026-01-22 03:29:29,640 - DEBUG - Route Targets: ['target:64512:8000008'] 2026-01-22 03:29:29,641 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,650 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/b80b2685-4c49-4beb-87fa-1aa488869f20 2026-01-22 03:29:29,668 - INFO - Verified VN network id 16 for VN 0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,669 - INFO - Verifications in API Server for VN ctest-vn2-58807010 passed 2026-01-22 03:29:29,669 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,678 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/b80b2685-4c49-4beb-87fa-1aa488869f20 2026-01-22 03:29:29,700 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/b80b2685-4c49-4beb-87fa-1aa488869f20 2026-01-22 03:29:29,712 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/3d5cf819-d036-4697-a61f-d0aa32d09c1f 2026-01-22 03:29:29,731 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '434875019300391598', 'uuid-lslong': '13583248881062121763'}, 'enable': 'true', 'created': '2026-01-22T03:29:29', 'last-modified': '2026-01-22T03:29:29', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.448799'} 2026-01-22 03:29:29,739 - DEBUG - Route Targets: ['target:64512:8000008'] 2026-01-22 03:29:29,746 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '434875019300391598', 'uuid-lslong': '13583248881062121763'}, 'enable': 'true', 'created': '2026-01-22T03:29:29', 'last-modified': '2026-01-22T03:29:29', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.461910'} 2026-01-22 03:29:29,753 - DEBUG - Route Targets: ['target:64512:8000008'] 2026-01-22 03:29:29,758 - DEBUG - Control-node 10.0.0.131 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN0-95493504', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '434875019300391598', 'uuid-lslong': '13583248881062121763'}, 'enable': 'true', 'created': '2026-01-22T03:29:29', 'last-modified': '2026-01-22T03:29:29', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.477195'} 2026-01-22 03:29:29,763 - DEBUG - Route Targets: ['target:64512:8000008'] 2026-01-22 03:29:29,763 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-58807010 passed 2026-01-22 03:29:29,764 - DEBUG - ====Verifying policy data for ctest-vn2-58807010 in API_Server ====== 2026-01-22 03:29:29,764 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:29,771 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:29,789 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:29,888 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/0608fc86-a717-4aae-bc81-64e3e36e1123 2026-01-22 03:29:29,899 - DEBUG - =>VN ctest-vn2-58807010 has no policy to be verified 2026-01-22 03:29:29,899 - DEBUG - Verifying the vn in opserver 2026-01-22 03:29:29,899 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010 virtual network link through opserver 10.0.0.38 2026-01-22 03:29:29,900 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2026-01-22 03:29:29,915 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010?flat'} 2026-01-22 03:29:29,915 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010 virtual network link through opserver 10.0.0.50 2026-01-22 03:29:29,915 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2026-01-22 03:29:29,982 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010?flat'} 2026-01-22 03:29:29,982 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010 virtual network link through opserver 10.0.0.131 2026-01-22 03:29:29,982 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-networks 2026-01-22 03:29:29,996 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010', 'href': 'http://10.0.0.131:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010?flat'} 2026-01-22 03:29:29,996 - INFO - Validated that VN default-domain:ctest-TestBasicVMVN0-95493504:ctest-vn2-58807010 is found in opserver 2026-01-22 03:29:30,020 - DEBUG - Do not have enough data to verify VN in agent 2026-01-22 03:29:30,032 - DEBUG - VRF ids for VN ctest-vn2-58807010: {} 2026-01-22 03:29:31,950 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-1) 2026-01-22 03:29:33,564 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5055-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5055-2) 2026-01-22 03:29:33,564 - INFO - Waiting for VM ctest-vn1_vm1-02397652 to be up.. 2026-01-22 03:29:33,633 - DEBUG - VM is in ERROR state now 2026-01-22 03:29:33,633 - ERROR - VM in error state. 2026-01-22 03:29:33,633 - ERROR - VM is not in ACTIVE state 2026-01-22 03:29:33,720 - ERROR - Fatal Nova Exception while getting VM detail 2026-01-22 03:29:33,721 - DEBUG - None 2026-01-22 03:29:33,722 - INFO - Skip interface_detach for VM ctest-vn2_vm1-22905263 in state BUILD 2026-01-22 03:29:33,723 - INFO - Deleting VM ctest-vn2_vm1-22905263 2026-01-22 03:29:33,723 - ERROR - VM ctest-vn2_vm1-22905263 has failed to come up 2026-01-22 03:29:33,724 - ERROR - Nova failed to get host of the VM 2026-01-22 03:29:33,724 - ERROR - VM ctest-vn2_vm1-22905263 has failed to come up 2026-01-22 03:29:33,724 - ERROR - Nova failed to get host of the VM 2026-01-22 03:29:33,795 - INFO - Skip interface_detach for VM ctest-vn1_vm1-02397652 in state ERROR 2026-01-22 03:29:33,796 - INFO - Deleting VM ctest-vn1_vm1-02397652 2026-01-22 03:29:33,797 - ERROR - VM ctest-vn1_vm1-02397652 has failed to come up 2026-01-22 03:29:33,797 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2026-01-22T03:29:31Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2026-01-22 03:29:34,057 - INFO - Deleting VN ctest-vn2-58807010 2026-01-22 03:29:34,127 - DEBUG - RefsExistError Delete when resource still referred: ['http://10.0.0.131:8082/virtual-machine-interface/6ac8d214-7b8a-40d3-92e7-642f743970f7'] while deleting VN ctest-vn2-58807010..Will retry 2026-01-22 03:29:36,143 - DEBUG - RefsExistError Delete when resource still referred: ['http://10.0.0.131:8082/instance-ip/9d699453-6b85-4b0c-8a09-bacc20d78538'] while deleting VN ctest-vn2-58807010..Will retry 2026-01-22 03:29:38,287 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/b80b2685-4c49-4beb-87fa-1aa488869f20 2026-01-22 03:29:38,293 - DEBUG - Response Code: 404 2026-01-22 03:29:38,293 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:38,298 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:38,324 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:38,444 - INFO - Validated that VN ctest-vn2-58807010 is not found in API Server 2026-01-22 03:29:38,458 - DEBUG - VN ctest-vn2-58807010 is not present in Agent 10.0.0.23 2026-01-22 03:29:38,469 - DEBUG - VN ctest-vn2-58807010 is not present in Agent 10.0.0.141 2026-01-22 03:29:38,469 - INFO - Validated that VN ctest-vn2-58807010 is not in any agent 2026-01-22 03:29:38,499 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn2-58807010 info 2026-01-22 03:29:38,675 - INFO - Deleting VN ctest-vn1-11629845 2026-01-22 03:29:38,802 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/ef356bcd-6022-41b9-9e4d-d1a738dc425e 2026-01-22 03:29:38,808 - DEBUG - Response Code: 404 2026-01-22 03:29:38,808 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-01-22 03:29:38,819 - DEBUG - Requesting: http://10.0.0.38:8082/domain/43eea14e-faa6-4579-85d2-09c8cc682740 2026-01-22 03:29:38,836 - DEBUG - Requesting: http://10.0.0.38:8082/project/f95967f5-10b0-498a-b932-01ed7d206c1e 2026-01-22 03:29:38,944 - INFO - Validated that VN ctest-vn1-11629845 is not found in API Server 2026-01-22 03:29:38,955 - DEBUG - VN ctest-vn1-11629845 is not present in Agent 10.0.0.23 2026-01-22 03:29:38,968 - DEBUG - VN ctest-vn1-11629845 is not present in Agent 10.0.0.141 2026-01-22 03:29:38,968 - INFO - Validated that VN ctest-vn1-11629845 is not in any agent 2026-01-22 03:29:39,001 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn1-11629845 info 2026-01-22 03:29:39,039 - INFO - Deleted policy policy2 2026-01-22 03:29:39,081 - INFO - Deleted policy policy1 2026-01-22 03:29:39,763 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Thu Jan 22 03:29:33 2026 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/vm_regression/test_vm_serial.py in test_process_restart_in_policy_between_vns(self=) 404 vm1_fixture = self.create_vm(vn1_fixture, vn1_vm1_name) 405 vm2_fixture = self.create_vm(vn2_fixture, vn2_vm1_name) 406 assert vm1_fixture.wait_till_vm_is_up() 407 assert vm2_fixture.wait_till_vm_is_up() 408 assert vm1_fixture.ping_with_certainty(vm2_fixture.vm_ip) vm1_fixture = vm1_fixture.wait_till_vm_is_up = > AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/vm_regression/test_vm_serial.py", line 406, in test_process_restart_in_policy_between_vns assert vm1_fixture.wait_till_vm_is_up() AssertionError 2026-01-22 03:29:39,763 - DEBUG - Skipping xmpp flap check 2026-01-22 03:29:39,763 - INFO -
2026-01-22 03:29:39,763 - INFO - END TEST : test_process_restart_in_policy_between_vns : FAILED[0:00:13]
2026-01-22 03:29:39,763 - INFO - -------------------------------------------------------------------------------- 2026-01-22 03:29:39,766 - INFO - ================================================================================ 2026-01-22 03:29:39,766 - INFO - STARTING TEST : test_underlay_broadcast_traffic_handling 2026-01-22 03:29:39,766 - INFO - TEST DESCRIPTION : Test the underlay brocast traffic handling by vrouter. (Bug-1545229). 1. Send broadcast traffic from one compute node. 2. Other compute in same subnet should receive that traffic. 3. Receiving compute should treat this traffic as underlay. 4. Compute should not replicate the packet and send the copy back. Pass criteria: Step 3-4 should pass Maintainer : chhandak@juniper.net 2026-01-22 03:29:40,429 - DEBUG - Skipping xmpp flap check 2026-01-22 03:29:40,430 - INFO - Initial checks done. Running the testcase now 2026-01-22 03:29:40,430 - INFO - 2026-01-22 03:29:40,638 - INFO - Agent interface name: ens6 2026-01-22 03:29:40,724 - DEBUG - Executing command: sudo tcpdump -nni ens6 -U host 10.20.0.255 -w /tmp/ens6_ctest-random-62497176.pcap 2026-01-22 03:29:46,141 - INFO - Pinging broacast address 10.20.0.255 from compute 10.20.0.131 2026-01-22 03:30:00,420 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-01-22 03:30:02,485 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-62497176.pcap | grep -c length 2026-01-22 03:30:02,502 - DEBUG - STDOUT: 10 2026-01-22 03:30:02,503 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-62497176.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-01-22 03:30:02,503 - INFO - 10 packets are found in tcpdump output as expected 2026-01-22 03:30:02,503 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-01-22 03:30:04,547 - INFO - Packet count matched: Compute 10.0.0.141 has receive only 10 packet from source IP 10.20.0.255. No duplicate packet seen 2026-01-22 03:30:05,240 - DEBUG - Skipping xmpp flap check
2026-01-22 03:30:05,241 - INFO - END TEST : test_underlay_broadcast_traffic_handling : PASSED[0:00:26]
2026-01-22 03:30:05,241 - INFO - -------------------------------------------------------------------------------- 2026-01-22 03:30:06,059 - INFO - Deleted project: ctest-TestBasicVMVN0-95493504, ID : f95967f5-10b0-498a-b932-01ed7d206c1e