2025-07-22 16:44:53,692 - INFO - Domain Default found not creating 2025-07-22 16:44:53,841 - INFO - Project ctest-TestSvcHC-12258299 not found, creating it 2025-07-22 16:44:54,371 - INFO - Created Project:ctest-TestSvcHC-12258299, ID : 70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:44:55,846 - INFO - ================================================================================ 2025-07-22 16:44:55,846 - INFO - STARTING TEST : test_svc_hc_basic 2025-07-22 16:44:57,114 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-07-22 16:44:57,114 - INFO - Initial checks done. Running the testcase now 2025-07-22 16:44:57,114 - INFO - 2025-07-22 16:44:57,831 - DEBUG - Response for create_network : {'network': {'id': 'b8f6f30d-55bc-49ee-a482-5ad3de96f492', 'name': 'ctest-mgmt_vn-99004648', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-mgmt_vn-99004648'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:44:58,093 - DEBUG - Response for create_subnet : {'subnet': {'id': 'aca3b02a-7b68-4524-8c77-6ed070219249', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': 'b8f6f30d-55bc-49ee-a482-5ad3de96f492', 'ip_version': 4, 'cidr': '67.204.129.0/26', 'allocation_pools': [{'start': '67.204.129.2', 'end': '67.204.129.62'}], 'gateway_ip': '67.204.129.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '67.204.129.2', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:44:58,118 - INFO - Created VN ctest-mgmt_vn-99004648 2025-07-22 16:44:58,137 - DEBUG - VN ctest-mgmt_vn-99004648 UUID is b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,216 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:44:58,506 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:44:58,536 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:44:58,553 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:44:58,649 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,662 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,672 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/ff2e997e-8149-415a-97cb-7b79d0db0711 2025-07-22 16:44:58,680 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/ff2e997e-8149-415a-97cb-7b79d0db0711 2025-07-22 16:44:58,689 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/acb2e07d-02ca-4cad-bd12-66e326953ea6 2025-07-22 16:44:58,697 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:44:58,698 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,710 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/ff2e997e-8149-415a-97cb-7b79d0db0711 2025-07-22 16:44:58,729 - INFO - Verified VN network id 12 for VN b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,729 - INFO - Verifications in API Server for VN ctest-mgmt_vn-99004648 passed 2025-07-22 16:44:58,729 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,740 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/ff2e997e-8149-415a-97cb-7b79d0db0711 2025-07-22 16:44:58,748 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/ff2e997e-8149-415a-97cb-7b79d0db0711 2025-07-22 16:44:58,756 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/acb2e07d-02ca-4cad-bd12-66e326953ea6 2025-07-22 16:44:58,770 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13328107385848089070', 'uuid-lslong': '11854137035211601042'}, 'enable': 'true', 'created': '2025-07-22T16:44:57', 'last-modified': '2025-07-22T16:44:58', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.732343'} 2025-07-22 16:44:58,777 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:44:58,783 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13328107385848089070', 'uuid-lslong': '11854137035211601042'}, 'enable': 'true', 'created': '2025-07-22T16:44:57', 'last-modified': '2025-07-22T16:44:58', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.750289'} 2025-07-22 16:44:58,790 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:44:58,799 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13328107385848089070', 'uuid-lslong': '11854137035211601042'}, 'enable': 'true', 'created': '2025-07-22T16:44:57', 'last-modified': '2025-07-22T16:44:58', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.759515'} 2025-07-22 16:44:58,807 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:44:58,807 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-99004648 passed 2025-07-22 16:44:58,808 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-99004648 in API_Server ====== 2025-07-22 16:44:58,808 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:44:58,816 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:44:58,828 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:44:58,925 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/b8f6f30d-55bc-49ee-a482-5ad3de96f492 2025-07-22 16:44:58,936 - DEBUG - =>VN ctest-mgmt_vn-99004648 has no policy to be verified 2025-07-22 16:44:58,936 - DEBUG - Verifying the vn in opserver 2025-07-22 16:44:58,936 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648 virtual network link through opserver 10.0.0.254 2025-07-22 16:44:58,936 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:44:59,231 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:44:59,294 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648?flat'} 2025-07-22 16:44:59,294 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648 virtual network link through opserver 10.0.0.50 2025-07-22 16:44:59,294 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:44:59,595 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:44:59,671 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648?flat'} 2025-07-22 16:44:59,671 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648 virtual network link through opserver 10.0.0.23 2025-07-22 16:44:59,671 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:44:59,970 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:45:00,038 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648?flat'} 2025-07-22 16:45:00,038 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-99004648 is found in opserver 2025-07-22 16:45:00,069 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:45:00,085 - DEBUG - VRF ids for VN ctest-mgmt_vn-99004648: {} 2025-07-22 16:45:00,375 - DEBUG - Response for create_network : {'network': {'id': '49376377-c6d5-40c4-aa9e-5cdffc815ace', 'name': 'ctest-left_vn-86212814', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-left_vn-86212814'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:45:00,615 - DEBUG - Response for create_subnet : {'subnet': {'id': '2296ddd5-2e30-48dd-9a82-4ebbc0312fc3', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': '49376377-c6d5-40c4-aa9e-5cdffc815ace', 'ip_version': 4, 'cidr': '186.33.35.192/26', 'allocation_pools': [{'start': '186.33.35.194', 'end': '186.33.35.254'}], 'gateway_ip': '186.33.35.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '186.33.35.194', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:45:00,637 - INFO - Created VN ctest-left_vn-86212814 2025-07-22 16:45:00,651 - DEBUG - VN ctest-left_vn-86212814 UUID is 49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:00,731 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:45:00,738 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:45:00,753 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:45:00,849 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:00,862 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:00,873 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/96b06374-5914-4a50-94df-d2f502d884d1 2025-07-22 16:45:00,883 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/96b06374-5914-4a50-94df-d2f502d884d1 2025-07-22 16:45:00,891 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/48850d41-8290-4858-9fce-3409d48508cd 2025-07-22 16:45:00,899 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:45:00,900 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:00,909 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/96b06374-5914-4a50-94df-d2f502d884d1 2025-07-22 16:45:00,926 - INFO - Verified VN network id 13 for VN 49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:00,927 - INFO - Verifications in API Server for VN ctest-left_vn-86212814 passed 2025-07-22 16:45:00,927 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:00,938 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/96b06374-5914-4a50-94df-d2f502d884d1 2025-07-22 16:45:00,946 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/96b06374-5914-4a50-94df-d2f502d884d1 2025-07-22 16:45:00,955 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/48850d41-8290-4858-9fce-3409d48508cd 2025-07-22 16:45:00,967 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5275794854575947972', 'uuid-lslong': '12294366149851830990'}, 'enable': 'true', 'created': '2025-07-22T16:45:00', 'last-modified': '2025-07-22T16:45:00', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.410565'} 2025-07-22 16:45:00,974 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:45:00,980 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5275794854575947972', 'uuid-lslong': '12294366149851830990'}, 'enable': 'true', 'created': '2025-07-22T16:45:00', 'last-modified': '2025-07-22T16:45:00', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.422102'} 2025-07-22 16:45:00,988 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:45:00,996 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5275794854575947972', 'uuid-lslong': '12294366149851830990'}, 'enable': 'true', 'created': '2025-07-22T16:45:00', 'last-modified': '2025-07-22T16:45:00', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.439662'} 2025-07-22 16:45:01,003 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:45:01,003 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-86212814 passed 2025-07-22 16:45:01,004 - DEBUG - ====Verifying policy data for ctest-left_vn-86212814 in API_Server ====== 2025-07-22 16:45:01,004 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:45:01,013 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:45:01,027 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:45:01,141 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/49376377-c6d5-40c4-aa9e-5cdffc815ace 2025-07-22 16:45:01,154 - DEBUG - =>VN ctest-left_vn-86212814 has no policy to be verified 2025-07-22 16:45:01,154 - DEBUG - Verifying the vn in opserver 2025-07-22 16:45:01,154 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814 virtual network link through opserver 10.0.0.254 2025-07-22 16:45:01,154 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:45:01,219 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814?flat'} 2025-07-22 16:45:01,220 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814 virtual network link through opserver 10.0.0.50 2025-07-22 16:45:01,220 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:45:01,278 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814?flat'} 2025-07-22 16:45:01,278 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814 virtual network link through opserver 10.0.0.23 2025-07-22 16:45:01,278 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:45:01,336 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814?flat'} 2025-07-22 16:45:01,336 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-86212814 is found in opserver 2025-07-22 16:45:01,365 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:45:01,377 - DEBUG - VRF ids for VN ctest-left_vn-86212814: {} 2025-07-22 16:45:01,681 - DEBUG - Response for create_network : {'network': {'id': '7001b908-78f4-400a-ac80-d75a15987c17', 'name': 'ctest-right_vn-31799893', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-right_vn-31799893'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:45:01,982 - DEBUG - Response for create_subnet : {'subnet': {'id': '89420ba7-76c0-4746-bde0-f67a61619d35', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': '7001b908-78f4-400a-ac80-d75a15987c17', 'ip_version': 4, 'cidr': '52.161.63.192/26', 'allocation_pools': [{'start': '52.161.63.194', 'end': '52.161.63.254'}], 'gateway_ip': '52.161.63.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '52.161.63.194', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:45:02,009 - INFO - Created VN ctest-right_vn-31799893 2025-07-22 16:45:02,024 - DEBUG - VN ctest-right_vn-31799893 UUID is 7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,110 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:45:02,117 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:45:02,131 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:45:02,230 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,245 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,257 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/f5419d50-2581-44f2-97a6-81513b42c329 2025-07-22 16:45:02,266 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/f5419d50-2581-44f2-97a6-81513b42c329 2025-07-22 16:45:02,275 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/59603352-13c4-439f-8071-6f198c0b78d5 2025-07-22 16:45:02,284 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:45:02,284 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,296 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/f5419d50-2581-44f2-97a6-81513b42c329 2025-07-22 16:45:02,316 - INFO - Verified VN network id 14 for VN 7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,316 - INFO - Verifications in API Server for VN ctest-right_vn-31799893 passed 2025-07-22 16:45:02,316 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,327 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/f5419d50-2581-44f2-97a6-81513b42c329 2025-07-22 16:45:02,336 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/f5419d50-2581-44f2-97a6-81513b42c329 2025-07-22 16:45:02,345 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/59603352-13c4-439f-8071-6f198c0b78d5 2025-07-22 16:45:02,359 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8070935453264789514', 'uuid-lslong': '12430171753451912215'}, 'enable': 'true', 'created': '2025-07-22T16:45:01', 'last-modified': '2025-07-22T16:45:01', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.437738'} 2025-07-22 16:45:02,367 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:45:02,374 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8070935453264789514', 'uuid-lslong': '12430171753451912215'}, 'enable': 'true', 'created': '2025-07-22T16:45:01', 'last-modified': '2025-07-22T16:45:01', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.450265'} 2025-07-22 16:45:02,381 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:45:02,388 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8070935453264789514', 'uuid-lslong': '12430171753451912215'}, 'enable': 'true', 'created': '2025-07-22T16:45:01', 'last-modified': '2025-07-22T16:45:01', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.460524'} 2025-07-22 16:45:02,396 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:45:02,396 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-31799893 passed 2025-07-22 16:45:02,396 - DEBUG - ====Verifying policy data for ctest-right_vn-31799893 in API_Server ====== 2025-07-22 16:45:02,396 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:45:02,403 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:45:02,417 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:45:02,526 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/7001b908-78f4-400a-ac80-d75a15987c17 2025-07-22 16:45:02,538 - DEBUG - =>VN ctest-right_vn-31799893 has no policy to be verified 2025-07-22 16:45:02,538 - DEBUG - Verifying the vn in opserver 2025-07-22 16:45:02,538 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893 virtual network link through opserver 10.0.0.254 2025-07-22 16:45:02,539 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:45:02,615 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893?flat'} 2025-07-22 16:45:02,615 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893 virtual network link through opserver 10.0.0.50 2025-07-22 16:45:02,615 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:45:02,721 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893?flat'} 2025-07-22 16:45:02,721 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893 virtual network link through opserver 10.0.0.23 2025-07-22 16:45:02,721 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:45:02,787 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893?flat'} 2025-07-22 16:45:02,787 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-31799893 is found in opserver 2025-07-22 16:45:02,812 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:45:02,825 - DEBUG - VRF ids for VN ctest-right_vn-31799893: {} 2025-07-22 16:45:02,882 - DEBUG - Services list from nova: [, , , ] 2025-07-22 16:45:03,990 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 16:45:05,412 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1) 2025-07-22 16:45:05,413 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-58964242'] 2025-07-22 16:45:05,740 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-58964242'] created succesfully 2025-07-22 16:46:46,416 - DEBUG - Installing image tiny_nat_fw 2025-07-22 16:46:46,416 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-07-22 16:46:46,416 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-22 16:46:46,416 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-22 16:46:46,416 - DEBUG - Unzip image 2025-07-22 16:46:46,416 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-22 16:46:46,434 - DEBUG - Try to create image in user context 2025-07-22 16:46:46,497 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-58964242'] 2025-07-22 16:46:46,600 - INFO - Service template: ['default-domain', 'ctest-service_template_1-58964242'] deleted successfully. 2025-07-22 16:46:46,600 - INFO - Skip interface_detach for VM ctest-right_vm-00105891 in state BUILD 2025-07-22 16:46:46,600 - INFO - Deleting VM ctest-right_vm-00105891 2025-07-22 16:46:46,627 - ERROR - VM ctest-right_vm-00105891 has failed to come up 2025-07-22 16:46:46,627 - ERROR - Nova failed to get host of the VM 2025-07-22 16:46:46,627 - ERROR - VM ctest-right_vm-00105891 has failed to come up 2025-07-22 16:46:46,628 - ERROR - Nova failed to get host of the VM 2025-07-22 16:46:46,687 - INFO - Skip interface_detach for VM ctest-left_vm-18177056 in state BUILD 2025-07-22 16:46:46,688 - INFO - Deleting VM ctest-left_vm-18177056 2025-07-22 16:46:46,688 - ERROR - VM ctest-left_vm-18177056 has failed to come up 2025-07-22 16:46:46,688 - ERROR - Nova failed to get host of the VM 2025-07-22 16:46:46,688 - ERROR - VM ctest-left_vm-18177056 has failed to come up 2025-07-22 16:46:46,689 - ERROR - Nova failed to get host of the VM 2025-07-22 16:46:46,750 - INFO - Deleting VN ctest-right_vn-31799893 2025-07-22 16:46:46,795 - DEBUG - VN 7001b908-78f4-400a-ac80-d75a15987c17 still in use: Unable to complete operation on network 7001b908-78f4-400a-ac80-d75a15987c17. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-4d4aab59-2611-454c-b6ff-7a881534fd0a'] 2025-07-22 16:46:46,796 - WARNING - Deleting VN ctest-right_vn-31799893 failed..Will retry 2025-07-22 16:46:48,978 - DEBUG - Response for deleting network () 2025-07-22 16:46:48,978 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/f5419d50-2581-44f2-97a6-81513b42c329 2025-07-22 16:46:48,986 - DEBUG - Response Code: 404 2025-07-22 16:46:48,986 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:48,993 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:49,022 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:49,120 - INFO - Validated that VN ctest-right_vn-31799893 is not found in API Server 2025-07-22 16:46:49,133 - DEBUG - VN ctest-right_vn-31799893 is not present in Agent 10.0.0.19 2025-07-22 16:46:49,147 - DEBUG - VN ctest-right_vn-31799893 is not present in Agent 10.0.0.27 2025-07-22 16:46:49,147 - INFO - Validated that VN ctest-right_vn-31799893 is not in any agent 2025-07-22 16:46:49,186 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-31799893 info 2025-07-22 16:46:49,186 - INFO - Deleting VN ctest-left_vn-86212814 2025-07-22 16:46:49,420 - DEBUG - Response for deleting network () 2025-07-22 16:46:49,420 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/96b06374-5914-4a50-94df-d2f502d884d1 2025-07-22 16:46:49,432 - DEBUG - Response Code: 404 2025-07-22 16:46:49,432 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:49,444 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:49,460 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:49,554 - INFO - Validated that VN ctest-left_vn-86212814 is not found in API Server 2025-07-22 16:46:49,570 - DEBUG - VN ctest-left_vn-86212814 is not present in Agent 10.0.0.19 2025-07-22 16:46:49,585 - DEBUG - VN ctest-left_vn-86212814 is not present in Agent 10.0.0.27 2025-07-22 16:46:49,585 - INFO - Validated that VN ctest-left_vn-86212814 is not in any agent 2025-07-22 16:46:49,633 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-86212814 info 2025-07-22 16:46:49,633 - INFO - Deleting VN ctest-mgmt_vn-99004648 2025-07-22 16:46:49,793 - DEBUG - Response for deleting network () 2025-07-22 16:46:49,793 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/ff2e997e-8149-415a-97cb-7b79d0db0711 2025-07-22 16:46:49,800 - DEBUG - Response Code: 404 2025-07-22 16:46:49,800 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:49,819 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:49,880 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:49,976 - INFO - Validated that VN ctest-mgmt_vn-99004648 is not found in API Server 2025-07-22 16:46:49,995 - DEBUG - VN ctest-mgmt_vn-99004648 is not present in Agent 10.0.0.19 2025-07-22 16:46:50,010 - DEBUG - VN ctest-mgmt_vn-99004648 is not present in Agent 10.0.0.27 2025-07-22 16:46:50,010 - INFO - Validated that VN ctest-mgmt_vn-99004648 is not in any agent 2025-07-22 16:46:50,049 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-99004648 info 2025-07-22 16:46:51,306 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 16, in test_svc_hc_basic
    create_svms=True, max_inst=1)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network-nat.qcow2'
2025-07-22 16:46:51,324 - DEBUG - No XMPP flaps were noticed during the test 2025-07-22 16:46:51,324 - INFO -
2025-07-22 16:46:51,324 - INFO - END TEST : test_svc_hc_basic : FAILED[0:01:56]
2025-07-22 16:46:51,324 - INFO - -------------------------------------------------------------------------------- 2025-07-22 16:46:51,328 - INFO - ================================================================================ 2025-07-22 16:46:51,328 - INFO - STARTING TEST : test_svc_trans_segment_right_hc_fail_svm 2025-07-22 16:46:52,603 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-07-22 16:46:52,603 - INFO - Initial checks done. Running the testcase now 2025-07-22 16:46:52,603 - INFO - 2025-07-22 16:46:52,928 - DEBUG - Response for create_network : {'network': {'id': 'a4d345f0-0310-4b66-a554-dcd3f334071e', 'name': 'ctest-mgmt_vn-17045729', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-mgmt_vn-17045729'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:46:53,207 - DEBUG - Response for create_subnet : {'subnet': {'id': '7ba21567-7fed-410f-bb54-daa8a7e909d5', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': 'a4d345f0-0310-4b66-a554-dcd3f334071e', 'ip_version': 4, 'cidr': '205.233.108.0/26', 'allocation_pools': [{'start': '205.233.108.2', 'end': '205.233.108.62'}], 'gateway_ip': '205.233.108.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '205.233.108.2', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:46:53,234 - INFO - Created VN ctest-mgmt_vn-17045729 2025-07-22 16:46:53,252 - DEBUG - VN ctest-mgmt_vn-17045729 UUID is a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,341 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:53,348 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:53,361 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:53,463 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,479 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,501 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/d56ab8c0-3edd-4cf9-a68c-fb8c40b85f55 2025-07-22 16:46:53,511 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/d56ab8c0-3edd-4cf9-a68c-fb8c40b85f55 2025-07-22 16:46:53,520 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/21be7e12-a9c2-48dd-82c8-9e267ed9505d 2025-07-22 16:46:53,529 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:46:53,529 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,543 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/d56ab8c0-3edd-4cf9-a68c-fb8c40b85f55 2025-07-22 16:46:53,562 - INFO - Verified VN network id 12 for VN a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,562 - INFO - Verifications in API Server for VN ctest-mgmt_vn-17045729 passed 2025-07-22 16:46:53,562 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,577 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/d56ab8c0-3edd-4cf9-a68c-fb8c40b85f55 2025-07-22 16:46:53,586 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/d56ab8c0-3edd-4cf9-a68c-fb8c40b85f55 2025-07-22 16:46:53,594 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/21be7e12-a9c2-48dd-82c8-9e267ed9505d 2025-07-22 16:46:53,608 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11876913539451997030', 'uuid-lslong': '11913389717178287902'}, 'enable': 'true', 'created': '2025-07-22T16:46:52', 'last-modified': '2025-07-22T16:46:53', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.475959'} 2025-07-22 16:46:53,614 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:46:53,621 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11876913539451997030', 'uuid-lslong': '11913389717178287902'}, 'enable': 'true', 'created': '2025-07-22T16:46:52', 'last-modified': '2025-07-22T16:46:53', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.491878'} 2025-07-22 16:46:53,628 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:46:53,635 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11876913539451997030', 'uuid-lslong': '11913389717178287902'}, 'enable': 'true', 'created': '2025-07-22T16:46:52', 'last-modified': '2025-07-22T16:46:53', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.503991'} 2025-07-22 16:46:53,642 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 16:46:53,643 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-17045729 passed 2025-07-22 16:46:53,643 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-17045729 in API_Server ====== 2025-07-22 16:46:53,643 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:53,650 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:53,664 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:53,772 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a4d345f0-0310-4b66-a554-dcd3f334071e 2025-07-22 16:46:53,783 - DEBUG - =>VN ctest-mgmt_vn-17045729 has no policy to be verified 2025-07-22 16:46:53,783 - DEBUG - Verifying the vn in opserver 2025-07-22 16:46:53,783 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729 virtual network link through opserver 10.0.0.254 2025-07-22 16:46:53,783 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:46:53,800 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729?flat'} 2025-07-22 16:46:53,800 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729 virtual network link through opserver 10.0.0.50 2025-07-22 16:46:53,800 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:46:53,817 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729?flat'} 2025-07-22 16:46:53,817 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729 virtual network link through opserver 10.0.0.23 2025-07-22 16:46:53,817 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:46:53,832 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729?flat'} 2025-07-22 16:46:53,832 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-mgmt_vn-17045729 is found in opserver 2025-07-22 16:46:53,859 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:46:53,874 - DEBUG - VRF ids for VN ctest-mgmt_vn-17045729: {} 2025-07-22 16:46:54,186 - DEBUG - Response for create_network : {'network': {'id': '3e55867c-2294-4080-9a68-a13d61d8334c', 'name': 'ctest-left_vn-17984978', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-left_vn-17984978'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:46:54,436 - DEBUG - Response for create_subnet : {'subnet': {'id': '753b76ed-7d0d-43a1-b585-a3b73a204d7a', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': '3e55867c-2294-4080-9a68-a13d61d8334c', 'ip_version': 4, 'cidr': '156.70.103.64/26', 'allocation_pools': [{'start': '156.70.103.66', 'end': '156.70.103.126'}], 'gateway_ip': '156.70.103.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '156.70.103.66', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:46:54,460 - INFO - Created VN ctest-left_vn-17984978 2025-07-22 16:46:54,476 - DEBUG - VN ctest-left_vn-17984978 UUID is 3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:54,571 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:54,582 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:54,598 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:54,704 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:54,719 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:54,731 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/36d3e0dd-015d-4a9a-a9c3-05fc1e42ebb0 2025-07-22 16:46:54,739 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/36d3e0dd-015d-4a9a-a9c3-05fc1e42ebb0 2025-07-22 16:46:54,749 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/598fed61-ea63-475e-a7e4-b0dd0d3a7117 2025-07-22 16:46:54,758 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:46:54,758 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:54,770 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/36d3e0dd-015d-4a9a-a9c3-05fc1e42ebb0 2025-07-22 16:46:54,790 - INFO - Verified VN network id 13 for VN 3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:54,790 - INFO - Verifications in API Server for VN ctest-left_vn-17984978 passed 2025-07-22 16:46:54,790 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:54,801 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/36d3e0dd-015d-4a9a-a9c3-05fc1e42ebb0 2025-07-22 16:46:54,810 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/36d3e0dd-015d-4a9a-a9c3-05fc1e42ebb0 2025-07-22 16:46:54,819 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/598fed61-ea63-475e-a7e4-b0dd0d3a7117 2025-07-22 16:46:54,836 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4491644071086145664', 'uuid-lslong': '11126320164425446220'}, 'enable': 'true', 'created': '2025-07-22T16:46:54', 'last-modified': '2025-07-22T16:46:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.482054'} 2025-07-22 16:46:54,844 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:46:54,852 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4491644071086145664', 'uuid-lslong': '11126320164425446220'}, 'enable': 'true', 'created': '2025-07-22T16:46:54', 'last-modified': '2025-07-22T16:46:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.491316'} 2025-07-22 16:46:54,860 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:46:54,872 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4491644071086145664', 'uuid-lslong': '11126320164425446220'}, 'enable': 'true', 'created': '2025-07-22T16:46:54', 'last-modified': '2025-07-22T16:46:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.518176'} 2025-07-22 16:46:54,881 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 16:46:54,881 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-17984978 passed 2025-07-22 16:46:54,881 - DEBUG - ====Verifying policy data for ctest-left_vn-17984978 in API_Server ====== 2025-07-22 16:46:54,881 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:54,889 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:54,903 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:55,004 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/3e55867c-2294-4080-9a68-a13d61d8334c 2025-07-22 16:46:55,018 - DEBUG - =>VN ctest-left_vn-17984978 has no policy to be verified 2025-07-22 16:46:55,018 - DEBUG - Verifying the vn in opserver 2025-07-22 16:46:55,018 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978 virtual network link through opserver 10.0.0.254 2025-07-22 16:46:55,018 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:46:55,034 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978?flat'} 2025-07-22 16:46:55,034 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978 virtual network link through opserver 10.0.0.50 2025-07-22 16:46:55,034 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:46:55,051 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978?flat'} 2025-07-22 16:46:55,051 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978 virtual network link through opserver 10.0.0.23 2025-07-22 16:46:55,051 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:46:55,068 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978?flat'} 2025-07-22 16:46:55,068 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-left_vn-17984978 is found in opserver 2025-07-22 16:46:55,096 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:46:55,110 - DEBUG - VRF ids for VN ctest-left_vn-17984978: {} 2025-07-22 16:46:55,427 - DEBUG - Response for create_network : {'network': {'id': 'ed330c8a-e96c-4240-a4bc-af4726d49cd1', 'name': 'ctest-right_vn-13987560', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-right_vn-13987560'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:46:55,695 - DEBUG - Response for create_subnet : {'subnet': {'id': 'fb9d4272-a970-4532-82e4-53079d66e4a2', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': 'ed330c8a-e96c-4240-a4bc-af4726d49cd1', 'ip_version': 4, 'cidr': '93.209.48.192/26', 'allocation_pools': [{'start': '93.209.48.194', 'end': '93.209.48.254'}], 'gateway_ip': '93.209.48.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '93.209.48.194', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:46:55,721 - INFO - Created VN ctest-right_vn-13987560 2025-07-22 16:46:55,735 - DEBUG - VN ctest-right_vn-13987560 UUID is ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:55,813 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:55,823 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:55,840 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:55,945 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:55,961 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:55,972 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/fc20e942-6b7e-458d-b349-fd2d490123fd 2025-07-22 16:46:55,985 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/fc20e942-6b7e-458d-b349-fd2d490123fd 2025-07-22 16:46:55,994 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/bc642e38-9219-4ce0-baf8-6daa49fb4a3c 2025-07-22 16:46:56,004 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:46:56,004 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:56,018 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/fc20e942-6b7e-458d-b349-fd2d490123fd 2025-07-22 16:46:56,039 - INFO - Verified VN network id 14 for VN ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:56,039 - INFO - Verifications in API Server for VN ctest-right_vn-13987560 passed 2025-07-22 16:46:56,040 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:56,053 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/fc20e942-6b7e-458d-b349-fd2d490123fd 2025-07-22 16:46:56,062 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/fc20e942-6b7e-458d-b349-fd2d490123fd 2025-07-22 16:46:56,071 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/bc642e38-9219-4ce0-baf8-6daa49fb4a3c 2025-07-22 16:46:56,088 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17092018801562370624', 'uuid-lslong': '11870555437970791633'}, 'enable': 'true', 'created': '2025-07-22T16:46:55', 'last-modified': '2025-07-22T16:46:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.454927'} 2025-07-22 16:46:56,096 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:46:56,104 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17092018801562370624', 'uuid-lslong': '11870555437970791633'}, 'enable': 'true', 'created': '2025-07-22T16:46:55', 'last-modified': '2025-07-22T16:46:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.478598'} 2025-07-22 16:46:56,111 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:46:56,117 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17092018801562370624', 'uuid-lslong': '11870555437970791633'}, 'enable': 'true', 'created': '2025-07-22T16:46:55', 'last-modified': '2025-07-22T16:46:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.488838'} 2025-07-22 16:46:56,124 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 16:46:56,124 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-13987560 passed 2025-07-22 16:46:56,125 - DEBUG - ====Verifying policy data for ctest-right_vn-13987560 in API_Server ====== 2025-07-22 16:46:56,125 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:56,134 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:56,158 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:56,282 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ed330c8a-e96c-4240-a4bc-af4726d49cd1 2025-07-22 16:46:56,297 - DEBUG - =>VN ctest-right_vn-13987560 has no policy to be verified 2025-07-22 16:46:56,297 - DEBUG - Verifying the vn in opserver 2025-07-22 16:46:56,297 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560 virtual network link through opserver 10.0.0.254 2025-07-22 16:46:56,297 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:46:56,313 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560?flat'} 2025-07-22 16:46:56,313 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560 virtual network link through opserver 10.0.0.50 2025-07-22 16:46:56,313 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:46:56,330 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560?flat'} 2025-07-22 16:46:56,331 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560 virtual network link through opserver 10.0.0.23 2025-07-22 16:46:56,331 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:46:56,346 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560?flat'} 2025-07-22 16:46:56,346 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-right_vn-13987560 is found in opserver 2025-07-22 16:46:56,373 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:46:56,394 - DEBUG - VRF ids for VN ctest-right_vn-13987560: {} 2025-07-22 16:46:56,725 - DEBUG - Response for create_network : {'network': {'id': '68364955-4233-4fac-b00e-08fba5d4f8ac', 'name': 'ctest-trans_left_vn-05730526', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-trans_left_vn-05730526'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:46:57,033 - DEBUG - Response for create_subnet : {'subnet': {'id': 'b45acd30-d3f0-444e-8693-024239d8cc10', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': '68364955-4233-4fac-b00e-08fba5d4f8ac', 'ip_version': 4, 'cidr': '201.115.239.128/26', 'allocation_pools': [{'start': '201.115.239.130', 'end': '201.115.239.190'}], 'gateway_ip': '201.115.239.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '201.115.239.130', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:46:57,060 - INFO - Created VN ctest-trans_left_vn-05730526 2025-07-22 16:46:57,078 - DEBUG - VN ctest-trans_left_vn-05730526 UUID is 68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,171 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:57,182 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:57,199 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:57,326 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,347 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,364 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/84e36a82-fbc6-40c8-a33c-5e4326ff6f67 2025-07-22 16:46:57,376 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/84e36a82-fbc6-40c8-a33c-5e4326ff6f67 2025-07-22 16:46:57,386 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/3966ed82-0bb3-4596-af4f-b952dc83698e 2025-07-22 16:46:57,396 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-22 16:46:57,396 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,410 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/84e36a82-fbc6-40c8-a33c-5e4326ff6f67 2025-07-22 16:46:57,434 - INFO - Verified VN network id 15 for VN 68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,434 - INFO - Verifications in API Server for VN ctest-trans_left_vn-05730526 passed 2025-07-22 16:46:57,434 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,447 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/84e36a82-fbc6-40c8-a33c-5e4326ff6f67 2025-07-22 16:46:57,458 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/84e36a82-fbc6-40c8-a33c-5e4326ff6f67 2025-07-22 16:46:57,471 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/3966ed82-0bb3-4596-af4f-b952dc83698e 2025-07-22 16:46:57,487 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7509270059218587564', 'uuid-lslong': '12686087077261277356'}, 'enable': 'true', 'created': '2025-07-22T16:46:56', 'last-modified': '2025-07-22T16:46:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.527390'} 2025-07-22 16:46:57,496 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-22 16:46:57,505 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7509270059218587564', 'uuid-lslong': '12686087077261277356'}, 'enable': 'true', 'created': '2025-07-22T16:46:56', 'last-modified': '2025-07-22T16:46:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.544375'} 2025-07-22 16:46:57,516 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-22 16:46:57,526 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7509270059218587564', 'uuid-lslong': '12686087077261277356'}, 'enable': 'true', 'created': '2025-07-22T16:46:56', 'last-modified': '2025-07-22T16:46:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.569215'} 2025-07-22 16:46:57,535 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-22 16:46:57,535 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_left_vn-05730526 passed 2025-07-22 16:46:57,535 - DEBUG - ====Verifying policy data for ctest-trans_left_vn-05730526 in API_Server ====== 2025-07-22 16:46:57,535 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:57,543 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:57,559 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:57,676 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/68364955-4233-4fac-b00e-08fba5d4f8ac 2025-07-22 16:46:57,689 - DEBUG - =>VN ctest-trans_left_vn-05730526 has no policy to be verified 2025-07-22 16:46:57,689 - DEBUG - Verifying the vn in opserver 2025-07-22 16:46:57,689 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526 virtual network link through opserver 10.0.0.254 2025-07-22 16:46:57,689 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:46:57,706 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526?flat'} 2025-07-22 16:46:57,706 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526 virtual network link through opserver 10.0.0.50 2025-07-22 16:46:57,706 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:46:57,722 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526?flat'} 2025-07-22 16:46:57,722 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526 virtual network link through opserver 10.0.0.23 2025-07-22 16:46:57,722 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:46:57,740 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526?flat'} 2025-07-22 16:46:57,740 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-trans_left_vn-05730526 is found in opserver 2025-07-22 16:46:57,766 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:46:57,781 - DEBUG - VRF ids for VN ctest-trans_left_vn-05730526: {} 2025-07-22 16:46:58,093 - DEBUG - Response for create_network : {'network': {'id': 'a9b43d39-5f3f-459a-ab44-057b3245c121', 'name': 'ctest-trans_right_vn-80842677', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'project_id': '70c2fea64f1c40d284b18e7d299df537', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-12258299', 'ctest-trans_right_vn-80842677'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 16:46:58,371 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e93d7347-d735-4ee4-8780-07ad98b06114', 'name': '', 'tenant_id': '70c2fea64f1c40d284b18e7d299df537', 'network_id': 'a9b43d39-5f3f-459a-ab44-057b3245c121', 'ip_version': 4, 'cidr': '77.240.228.192/26', 'allocation_pools': [{'start': '77.240.228.194', 'end': '77.240.228.254'}], 'gateway_ip': '77.240.228.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '77.240.228.194', 'tags': [], 'project_id': '70c2fea64f1c40d284b18e7d299df537'}} 2025-07-22 16:46:58,396 - INFO - Created VN ctest-trans_right_vn-80842677 2025-07-22 16:46:58,413 - DEBUG - VN ctest-trans_right_vn-80842677 UUID is a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,498 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:58,508 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:58,524 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:58,629 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,644 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,658 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/29ccf376-15f0-4318-8b8b-7bd00f684272 2025-07-22 16:46:58,668 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/29ccf376-15f0-4318-8b8b-7bd00f684272 2025-07-22 16:46:58,678 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/e56ebe98-9861-4a04-973e-f1fb1bc3eacd 2025-07-22 16:46:58,686 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-22 16:46:58,687 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,697 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/29ccf376-15f0-4318-8b8b-7bd00f684272 2025-07-22 16:46:58,716 - INFO - Verified VN network id 16 for VN a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,716 - INFO - Verifications in API Server for VN ctest-trans_right_vn-80842677 passed 2025-07-22 16:46:58,716 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,729 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/29ccf376-15f0-4318-8b8b-7bd00f684272 2025-07-22 16:46:58,737 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/29ccf376-15f0-4318-8b8b-7bd00f684272 2025-07-22 16:46:58,745 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/e56ebe98-9861-4a04-973e-f1fb1bc3eacd 2025-07-22 16:46:58,762 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12228466204838151578', 'uuid-lslong': '12340994905584550177'}, 'enable': 'true', 'created': '2025-07-22T16:46:58', 'last-modified': '2025-07-22T16:46:58', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.456350'} 2025-07-22 16:46:58,769 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-22 16:46:58,775 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12228466204838151578', 'uuid-lslong': '12340994905584550177'}, 'enable': 'true', 'created': '2025-07-22T16:46:58', 'last-modified': '2025-07-22T16:46:58', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.472236'} 2025-07-22 16:46:58,785 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-22 16:46:58,792 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-12258299', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12228466204838151578', 'uuid-lslong': '12340994905584550177'}, 'enable': 'true', 'created': '2025-07-22T16:46:58', 'last-modified': '2025-07-22T16:46:58', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.486053'} 2025-07-22 16:46:58,801 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-22 16:46:58,801 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_right_vn-80842677 passed 2025-07-22 16:46:58,801 - DEBUG - ====Verifying policy data for ctest-trans_right_vn-80842677 in API_Server ====== 2025-07-22 16:46:58,801 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:46:58,809 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:46:58,824 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:46:58,926 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/a9b43d39-5f3f-459a-ab44-057b3245c121 2025-07-22 16:46:58,937 - DEBUG - =>VN ctest-trans_right_vn-80842677 has no policy to be verified 2025-07-22 16:46:58,937 - DEBUG - Verifying the vn in opserver 2025-07-22 16:46:58,937 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677 virtual network link through opserver 10.0.0.254 2025-07-22 16:46:58,937 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 16:46:58,953 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677?flat'} 2025-07-22 16:46:58,953 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677 virtual network link through opserver 10.0.0.50 2025-07-22 16:46:58,953 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 16:46:58,969 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677?flat'} 2025-07-22 16:46:58,969 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677 virtual network link through opserver 10.0.0.23 2025-07-22 16:46:58,969 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 16:46:58,984 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677?flat'} 2025-07-22 16:46:58,984 - INFO - Validated that VN default-domain:ctest-TestSvcHC-12258299:ctest-trans_right_vn-80842677 is found in opserver 2025-07-22 16:46:59,009 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 16:46:59,022 - DEBUG - VRF ids for VN ctest-trans_right_vn-80842677: {} 2025-07-22 16:47:00,271 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1) 2025-07-22 16:47:01,687 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 16:47:01,688 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-21151116'] 2025-07-22 16:47:02,004 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-21151116'] created succesfully 2025-07-22 16:48:42,695 - DEBUG - Installing image tiny_trans_fw 2025-07-22 16:48:42,695 - DEBUG - Image info {'name': 'tinycore-net-transparent-fw.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-07-22 16:48:42,695 - DEBUG - Image build path file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-22 16:48:42,695 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-22 16:48:42,696 - DEBUG - Unzip image 2025-07-22 16:48:42,696 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-22 16:48:42,714 - DEBUG - Try to create image in user context 2025-07-22 16:48:42,766 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-21151116'] 2025-07-22 16:48:42,860 - INFO - Service template: ['default-domain', 'ctest-service_template_1-21151116'] deleted successfully. 2025-07-22 16:48:42,860 - INFO - Skip interface_detach for VM ctest-right_vm-28752337 in state BUILD 2025-07-22 16:48:42,860 - INFO - Deleting VM ctest-right_vm-28752337 2025-07-22 16:48:42,860 - ERROR - VM ctest-right_vm-28752337 has failed to come up 2025-07-22 16:48:42,860 - ERROR - Nova failed to get host of the VM 2025-07-22 16:48:42,860 - ERROR - VM ctest-right_vm-28752337 has failed to come up 2025-07-22 16:48:42,860 - ERROR - Nova failed to get host of the VM 2025-07-22 16:48:42,936 - INFO - Skip interface_detach for VM ctest-left_vm-45837700 in state BUILD 2025-07-22 16:48:42,936 - INFO - Deleting VM ctest-left_vm-45837700 2025-07-22 16:48:42,936 - ERROR - VM ctest-left_vm-45837700 has failed to come up 2025-07-22 16:48:42,936 - ERROR - Nova failed to get host of the VM 2025-07-22 16:48:42,936 - ERROR - VM ctest-left_vm-45837700 has failed to come up 2025-07-22 16:48:42,936 - ERROR - Nova failed to get host of the VM 2025-07-22 16:48:43,000 - INFO - Deleting VN ctest-trans_right_vn-80842677 2025-07-22 16:48:43,248 - DEBUG - Response for deleting network () 2025-07-22 16:48:43,248 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/29ccf376-15f0-4318-8b8b-7bd00f684272 2025-07-22 16:48:43,255 - DEBUG - Response Code: 404 2025-07-22 16:48:43,256 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:48:43,263 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:48:43,319 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:48:43,443 - INFO - Validated that VN ctest-trans_right_vn-80842677 is not found in API Server 2025-07-22 16:48:43,456 - DEBUG - VN ctest-trans_right_vn-80842677 is not present in Agent 10.0.0.19 2025-07-22 16:48:43,472 - DEBUG - VN ctest-trans_right_vn-80842677 is not present in Agent 10.0.0.27 2025-07-22 16:48:43,472 - INFO - Validated that VN ctest-trans_right_vn-80842677 is not in any agent 2025-07-22 16:48:43,542 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_right_vn-80842677 info 2025-07-22 16:48:43,542 - INFO - Deleting VN ctest-trans_left_vn-05730526 2025-07-22 16:48:43,817 - DEBUG - Response for deleting network () 2025-07-22 16:48:43,817 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/84e36a82-fbc6-40c8-a33c-5e4326ff6f67 2025-07-22 16:48:43,826 - DEBUG - Response Code: 404 2025-07-22 16:48:43,826 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:48:43,836 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:48:43,853 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:48:43,985 - INFO - Validated that VN ctest-trans_left_vn-05730526 is not found in API Server 2025-07-22 16:48:44,001 - DEBUG - VN ctest-trans_left_vn-05730526 is not present in Agent 10.0.0.19 2025-07-22 16:48:44,025 - DEBUG - VN ctest-trans_left_vn-05730526 is not present in Agent 10.0.0.27 2025-07-22 16:48:44,025 - INFO - Validated that VN ctest-trans_left_vn-05730526 is not in any agent 2025-07-22 16:48:44,086 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_left_vn-05730526 info 2025-07-22 16:48:44,086 - INFO - Deleting VN ctest-right_vn-13987560 2025-07-22 16:48:44,299 - DEBUG - Response for deleting network () 2025-07-22 16:48:44,299 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/fc20e942-6b7e-458d-b349-fd2d490123fd 2025-07-22 16:48:44,307 - DEBUG - Response Code: 404 2025-07-22 16:48:44,308 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:48:44,317 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:48:44,379 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:48:44,473 - INFO - Validated that VN ctest-right_vn-13987560 is not found in API Server 2025-07-22 16:48:44,486 - DEBUG - VN ctest-right_vn-13987560 is not present in Agent 10.0.0.19 2025-07-22 16:48:44,499 - DEBUG - VN ctest-right_vn-13987560 is not present in Agent 10.0.0.27 2025-07-22 16:48:44,499 - INFO - Validated that VN ctest-right_vn-13987560 is not in any agent 2025-07-22 16:48:44,538 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-13987560 info 2025-07-22 16:48:44,538 - INFO - Deleting VN ctest-left_vn-17984978 2025-07-22 16:48:44,736 - DEBUG - Response for deleting network () 2025-07-22 16:48:44,736 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/36d3e0dd-015d-4a9a-a9c3-05fc1e42ebb0 2025-07-22 16:48:44,743 - DEBUG - Response Code: 404 2025-07-22 16:48:44,743 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:48:44,759 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:48:44,775 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:48:44,881 - INFO - Validated that VN ctest-left_vn-17984978 is not found in API Server 2025-07-22 16:48:44,894 - DEBUG - VN ctest-left_vn-17984978 is not present in Agent 10.0.0.19 2025-07-22 16:48:44,907 - DEBUG - VN ctest-left_vn-17984978 is not present in Agent 10.0.0.27 2025-07-22 16:48:44,907 - INFO - Validated that VN ctest-left_vn-17984978 is not in any agent 2025-07-22 16:48:44,953 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-17984978 info 2025-07-22 16:48:44,953 - INFO - Deleting VN ctest-mgmt_vn-17045729 2025-07-22 16:48:45,150 - DEBUG - Response for deleting network () 2025-07-22 16:48:45,150 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/d56ab8c0-3edd-4cf9-a68c-fb8c40b85f55 2025-07-22 16:48:45,162 - DEBUG - Response Code: 404 2025-07-22 16:48:45,162 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 16:48:45,171 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 16:48:45,210 - DEBUG - Requesting: http://10.0.0.254:8082/project/70c2fea6-4f1c-40d2-84b1-8e7d299df537 2025-07-22 16:48:45,319 - INFO - Validated that VN ctest-mgmt_vn-17045729 is not found in API Server 2025-07-22 16:48:45,336 - DEBUG - VN ctest-mgmt_vn-17045729 is not present in Agent 10.0.0.19 2025-07-22 16:48:45,353 - DEBUG - VN ctest-mgmt_vn-17045729 is not present in Agent 10.0.0.27 2025-07-22 16:48:45,353 - INFO - Validated that VN ctest-mgmt_vn-17045729 is not in any agent 2025-07-22 16:48:45,404 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-17045729 info 2025-07-22 16:48:46,661 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 137, in test_svc_trans_segment_right_hc_fail_svm
    assert self.svc_hc_fail_svm_common()
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 180, in svc_hc_fail_svm_common
    create_svms=True, max_inst=1)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-net-transparent-fw.qcow2'
2025-07-22 16:48:46,677 - DEBUG - No XMPP flaps were noticed during the test 2025-07-22 16:48:46,678 - INFO -
2025-07-22 16:48:46,678 - INFO - END TEST : test_svc_trans_segment_right_hc_fail_svm : FAILED[0:01:55]
2025-07-22 16:48:46,678 - INFO - -------------------------------------------------------------------------------- 2025-07-22 16:48:47,497 - INFO - Deleted project: ctest-TestSvcHC-12258299, ID : 70c2fea6-4f1c-40d2-84b1-8e7d299df537