2025-07-23 16:35:14,395 - INFO - Domain Default found not creating 2025-07-23 16:35:14,548 - INFO - Project ctest-TestSvcHC-76392771 not found, creating it 2025-07-23 16:35:15,104 - INFO - Created Project:ctest-TestSvcHC-76392771, ID : 408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:16,606 - INFO - ================================================================================ 2025-07-23 16:35:16,606 - INFO - STARTING TEST : test_svc_hc_basic 2025-07-23 16:35:17,944 - DEBUG - Nothing to compare xmpp stats {'10.0.0.50': {'10.20.0.254': '0', '10.20.0.17': '0'}, '10.0.0.130': {'10.20.0.17': '0', '10.20.0.254': '0'}} with 2025-07-23 16:35:17,944 - INFO - Initial checks done. Running the testcase now 2025-07-23 16:35:17,944 - INFO - 2025-07-23 16:35:18,621 - DEBUG - Response for create_network : {'network': {'id': '63d0c9fa-bd21-4a0f-97d1-3d29f71f1338', 'name': 'ctest-mgmt_vn-82554464', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-mgmt_vn-82554464'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:35:18,962 - DEBUG - Response for create_subnet : {'subnet': {'id': 'f671b64c-5584-43e6-bc5b-c693b0dc2672', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': '63d0c9fa-bd21-4a0f-97d1-3d29f71f1338', 'ip_version': 4, 'cidr': '135.140.84.0/26', 'allocation_pools': [{'start': '135.140.84.2', 'end': '135.140.84.62'}], 'gateway_ip': '135.140.84.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '135.140.84.2', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:35:18,989 - INFO - Created VN ctest-mgmt_vn-82554464 2025-07-23 16:35:19,003 - DEBUG - VN ctest-mgmt_vn-82554464 UUID is 63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,089 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:19,396 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:19,422 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:35:19,443 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:19,545 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,561 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,573 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/7b574700-17da-4686-b7f8-940b6fe68421 2025-07-23 16:35:19,581 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/7b574700-17da-4686-b7f8-940b6fe68421 2025-07-23 16:35:19,588 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/1c5fc2cf-5ac3-4c19-a58c-2f95e2448e0d 2025-07-23 16:35:19,596 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:35:19,596 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,606 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/7b574700-17da-4686-b7f8-940b6fe68421 2025-07-23 16:35:19,623 - INFO - Verified VN network id 12 for VN 63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,623 - INFO - Verifications in API Server for VN ctest-mgmt_vn-82554464 passed 2025-07-23 16:35:19,623 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,637 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/7b574700-17da-4686-b7f8-940b6fe68421 2025-07-23 16:35:19,646 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/7b574700-17da-4686-b7f8-940b6fe68421 2025-07-23 16:35:19,655 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/1c5fc2cf-5ac3-4c19-a58c-2f95e2448e0d 2025-07-23 16:35:19,669 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7192470683662764559', 'uuid-lslong': '10939592220308607800'}, 'enable': 'true', 'created': '2025-07-23T16:35:18', 'last-modified': '2025-07-23T16:35:18', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.774441'} 2025-07-23 16:35:19,676 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:35:19,683 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7192470683662764559', 'uuid-lslong': '10939592220308607800'}, 'enable': 'true', 'created': '2025-07-23T16:35:18', 'last-modified': '2025-07-23T16:35:18', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.789204'} 2025-07-23 16:35:19,690 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:35:19,701 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7192470683662764559', 'uuid-lslong': '10939592220308607800'}, 'enable': 'true', 'created': '2025-07-23T16:35:18', 'last-modified': '2025-07-23T16:35:18', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.808110'} 2025-07-23 16:35:19,712 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:35:19,712 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-82554464 passed 2025-07-23 16:35:19,712 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-82554464 in API_Server ====== 2025-07-23 16:35:19,712 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:19,720 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:35:19,736 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:19,840 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/63d0c9fa-bd21-4a0f-97d1-3d29f71f1338 2025-07-23 16:35:19,853 - DEBUG - =>VN ctest-mgmt_vn-82554464 has no policy to be verified 2025-07-23 16:35:19,853 - DEBUG - Verifying the vn in opserver 2025-07-23 16:35:19,853 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464 virtual network link through opserver 10.0.0.23 2025-07-23 16:35:19,853 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:35:20,149 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:35:20,214 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464?flat'} 2025-07-23 16:35:20,214 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464 virtual network link through opserver 10.0.0.249 2025-07-23 16:35:20,214 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:35:20,523 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:35:20,586 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464?flat'} 2025-07-23 16:35:20,586 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464 virtual network link through opserver 10.0.0.73 2025-07-23 16:35:20,586 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:35:20,889 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:35:20,947 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464?flat'} 2025-07-23 16:35:20,947 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-82554464 is found in opserver 2025-07-23 16:35:20,988 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:35:21,008 - DEBUG - VRF ids for VN ctest-mgmt_vn-82554464: {} 2025-07-23 16:35:21,313 - DEBUG - Response for create_network : {'network': {'id': 'bdda9e99-14bd-453a-9fce-e6bddd7c5e34', 'name': 'ctest-left_vn-99607138', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-left_vn-99607138'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:35:21,578 - DEBUG - Response for create_subnet : {'subnet': {'id': '6a0f1d98-f483-49e1-a359-3b8a621c312e', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': 'bdda9e99-14bd-453a-9fce-e6bddd7c5e34', 'ip_version': 4, 'cidr': '123.181.199.128/26', 'allocation_pools': [{'start': '123.181.199.130', 'end': '123.181.199.190'}], 'gateway_ip': '123.181.199.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '123.181.199.130', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:35:21,602 - INFO - Created VN ctest-left_vn-99607138 2025-07-23 16:35:21,618 - DEBUG - VN ctest-left_vn-99607138 UUID is bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:21,702 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:21,709 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:35:21,723 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:21,826 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:21,839 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:21,851 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/b2899db3-1cee-45d1-8fe2-4d4130995742 2025-07-23 16:35:21,860 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/b2899db3-1cee-45d1-8fe2-4d4130995742 2025-07-23 16:35:21,869 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/2e6c8217-577b-473e-b2ee-7d498494321a 2025-07-23 16:35:21,877 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:35:21,878 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:21,889 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/b2899db3-1cee-45d1-8fe2-4d4130995742 2025-07-23 16:35:21,908 - INFO - Verified VN network id 13 for VN bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:21,908 - INFO - Verifications in API Server for VN ctest-left_vn-99607138 passed 2025-07-23 16:35:21,909 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:21,921 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/b2899db3-1cee-45d1-8fe2-4d4130995742 2025-07-23 16:35:21,929 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/b2899db3-1cee-45d1-8fe2-4d4130995742 2025-07-23 16:35:21,937 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/2e6c8217-577b-473e-b2ee-7d498494321a 2025-07-23 16:35:21,951 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13680421198406436154', 'uuid-lslong': '11515395000372059700'}, 'enable': 'true', 'created': '2025-07-23T16:35:21', 'last-modified': '2025-07-23T16:35:21', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.454156'} 2025-07-23 16:35:21,957 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:35:21,965 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13680421198406436154', 'uuid-lslong': '11515395000372059700'}, 'enable': 'true', 'created': '2025-07-23T16:35:21', 'last-modified': '2025-07-23T16:35:21', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.466980'} 2025-07-23 16:35:21,974 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:35:21,982 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13680421198406436154', 'uuid-lslong': '11515395000372059700'}, 'enable': 'true', 'created': '2025-07-23T16:35:21', 'last-modified': '2025-07-23T16:35:21', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.483042'} 2025-07-23 16:35:21,990 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:35:21,990 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-99607138 passed 2025-07-23 16:35:21,991 - DEBUG - ====Verifying policy data for ctest-left_vn-99607138 in API_Server ====== 2025-07-23 16:35:21,991 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:22,006 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:35:22,036 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:22,153 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/bdda9e99-14bd-453a-9fce-e6bddd7c5e34 2025-07-23 16:35:22,165 - DEBUG - =>VN ctest-left_vn-99607138 has no policy to be verified 2025-07-23 16:35:22,165 - DEBUG - Verifying the vn in opserver 2025-07-23 16:35:22,165 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138 virtual network link through opserver 10.0.0.23 2025-07-23 16:35:22,165 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:35:22,262 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138?flat'} 2025-07-23 16:35:22,262 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138 virtual network link through opserver 10.0.0.249 2025-07-23 16:35:22,262 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:35:22,379 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138?flat'} 2025-07-23 16:35:22,379 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138 virtual network link through opserver 10.0.0.73 2025-07-23 16:35:22,379 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:35:22,466 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138?flat'} 2025-07-23 16:35:22,466 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-99607138 is found in opserver 2025-07-23 16:35:22,504 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:35:22,523 - DEBUG - VRF ids for VN ctest-left_vn-99607138: {} 2025-07-23 16:35:22,816 - DEBUG - Response for create_network : {'network': {'id': '7a76b6e7-f568-4005-b8b0-419f5a3af6d2', 'name': 'ctest-right_vn-23861532', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-right_vn-23861532'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:35:23,060 - DEBUG - Response for create_subnet : {'subnet': {'id': '6791e694-da18-49fe-aab5-e6519cc28145', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': '7a76b6e7-f568-4005-b8b0-419f5a3af6d2', 'ip_version': 4, 'cidr': '146.120.65.0/26', 'allocation_pools': [{'start': '146.120.65.2', 'end': '146.120.65.62'}], 'gateway_ip': '146.120.65.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '146.120.65.2', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:35:23,100 - INFO - Created VN ctest-right_vn-23861532 2025-07-23 16:35:23,117 - DEBUG - VN ctest-right_vn-23861532 UUID is 7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,203 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:23,209 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:35:23,222 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:23,321 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,337 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,352 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/22696ac9-7ff7-424d-a6ce-5767393db5d6 2025-07-23 16:35:23,363 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/22696ac9-7ff7-424d-a6ce-5767393db5d6 2025-07-23 16:35:23,373 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/d3e2efe2-322a-4463-92dd-bfd272585e48 2025-07-23 16:35:23,382 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:35:23,382 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,395 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/22696ac9-7ff7-424d-a6ce-5767393db5d6 2025-07-23 16:35:23,414 - INFO - Verified VN network id 14 for VN 7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,414 - INFO - Verifications in API Server for VN ctest-right_vn-23861532 passed 2025-07-23 16:35:23,414 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,428 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/22696ac9-7ff7-424d-a6ce-5767393db5d6 2025-07-23 16:35:23,439 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/22696ac9-7ff7-424d-a6ce-5767393db5d6 2025-07-23 16:35:23,448 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/d3e2efe2-322a-4463-92dd-bfd272585e48 2025-07-23 16:35:23,462 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8824441627250016261', 'uuid-lslong': '13308209051549234898'}, 'enable': 'true', 'created': '2025-07-23T16:35:22', 'last-modified': '2025-07-23T16:35:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.477476'} 2025-07-23 16:35:23,469 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:35:23,475 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8824441627250016261', 'uuid-lslong': '13308209051549234898'}, 'enable': 'true', 'created': '2025-07-23T16:35:22', 'last-modified': '2025-07-23T16:35:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.489859'} 2025-07-23 16:35:23,482 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:35:23,494 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8824441627250016261', 'uuid-lslong': '13308209051549234898'}, 'enable': 'true', 'created': '2025-07-23T16:35:22', 'last-modified': '2025-07-23T16:35:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.508595'} 2025-07-23 16:35:23,504 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:35:23,505 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-23861532 passed 2025-07-23 16:35:23,505 - DEBUG - ====Verifying policy data for ctest-right_vn-23861532 in API_Server ====== 2025-07-23 16:35:23,505 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:35:23,513 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:35:23,526 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:35:23,632 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/7a76b6e7-f568-4005-b8b0-419f5a3af6d2 2025-07-23 16:35:23,643 - DEBUG - =>VN ctest-right_vn-23861532 has no policy to be verified 2025-07-23 16:35:23,643 - DEBUG - Verifying the vn in opserver 2025-07-23 16:35:23,643 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532 virtual network link through opserver 10.0.0.23 2025-07-23 16:35:23,643 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:35:23,708 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532?flat'} 2025-07-23 16:35:23,708 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532 virtual network link through opserver 10.0.0.249 2025-07-23 16:35:23,708 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:35:23,767 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532?flat'} 2025-07-23 16:35:23,767 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532 virtual network link through opserver 10.0.0.73 2025-07-23 16:35:23,767 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:35:23,834 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532?flat'} 2025-07-23 16:35:23,834 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-23861532 is found in opserver 2025-07-23 16:35:23,872 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:35:23,889 - DEBUG - VRF ids for VN ctest-right_vn-23861532: {} 2025-07-23 16:35:23,944 - DEBUG - Services list from nova: [, , , ] 2025-07-23 16:35:25,036 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3704-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3704-2) 2025-07-23 16:35:26,371 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3704-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3704-1) 2025-07-23 16:35:26,371 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-25337632'] 2025-07-23 16:35:26,681 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-25337632'] created succesfully 2025-07-23 16:37:07,400 - DEBUG - Installing image tiny_nat_fw 2025-07-23 16:37:07,400 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-07-23 16:37:07,400 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-23 16:37:07,401 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-23 16:37:07,401 - DEBUG - Unzip image 2025-07-23 16:37:07,401 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-23 16:37:07,421 - DEBUG - Try to create image in user context 2025-07-23 16:37:07,505 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-25337632'] 2025-07-23 16:37:07,616 - INFO - Service template: ['default-domain', 'ctest-service_template_1-25337632'] deleted successfully. 2025-07-23 16:37:07,617 - INFO - Skip interface_detach for VM ctest-right_vm-84321096 in state BUILD 2025-07-23 16:37:07,617 - INFO - Deleting VM ctest-right_vm-84321096 2025-07-23 16:37:07,652 - ERROR - VM ctest-right_vm-84321096 has failed to come up 2025-07-23 16:37:07,652 - ERROR - Nova failed to get host of the VM 2025-07-23 16:37:07,653 - ERROR - VM ctest-right_vm-84321096 has failed to come up 2025-07-23 16:37:07,653 - ERROR - Nova failed to get host of the VM 2025-07-23 16:37:07,718 - INFO - Skip interface_detach for VM ctest-left_vm-02922055 in state BUILD 2025-07-23 16:37:07,719 - INFO - Deleting VM ctest-left_vm-02922055 2025-07-23 16:37:07,719 - ERROR - VM ctest-left_vm-02922055 has failed to come up 2025-07-23 16:37:07,719 - ERROR - Nova failed to get host of the VM 2025-07-23 16:37:07,719 - ERROR - VM ctest-left_vm-02922055 has failed to come up 2025-07-23 16:37:07,719 - ERROR - Nova failed to get host of the VM 2025-07-23 16:37:07,785 - INFO - Deleting VN ctest-right_vn-23861532 2025-07-23 16:37:07,831 - DEBUG - VN 7a76b6e7-f568-4005-b8b0-419f5a3af6d2 still in use: Unable to complete operation on network 7a76b6e7-f568-4005-b8b0-419f5a3af6d2. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-ec7f2f25-481d-4c75-8118-b86296934b71'] 2025-07-23 16:37:07,831 - WARNING - Deleting VN ctest-right_vn-23861532 failed..Will retry 2025-07-23 16:37:10,044 - DEBUG - Response for deleting network () 2025-07-23 16:37:10,044 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/22696ac9-7ff7-424d-a6ce-5767393db5d6 2025-07-23 16:37:10,051 - DEBUG - Response Code: 404 2025-07-23 16:37:10,052 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:10,060 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:10,087 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:10,192 - INFO - Validated that VN ctest-right_vn-23861532 is not found in API Server 2025-07-23 16:37:10,207 - DEBUG - VN ctest-right_vn-23861532 is not present in Agent 10.0.0.50 2025-07-23 16:37:10,220 - DEBUG - VN ctest-right_vn-23861532 is not present in Agent 10.0.0.130 2025-07-23 16:37:10,220 - INFO - Validated that VN ctest-right_vn-23861532 is not in any agent 2025-07-23 16:37:10,259 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-23861532 info 2025-07-23 16:37:10,259 - INFO - Deleting VN ctest-left_vn-99607138 2025-07-23 16:37:10,457 - DEBUG - Response for deleting network () 2025-07-23 16:37:10,457 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/b2899db3-1cee-45d1-8fe2-4d4130995742 2025-07-23 16:37:10,473 - DEBUG - Response Code: 404 2025-07-23 16:37:10,473 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:10,489 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:10,507 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:10,612 - INFO - Validated that VN ctest-left_vn-99607138 is not found in API Server 2025-07-23 16:37:10,642 - DEBUG - VN ctest-left_vn-99607138 is not present in Agent 10.0.0.50 2025-07-23 16:37:10,657 - DEBUG - VN ctest-left_vn-99607138 is not present in Agent 10.0.0.130 2025-07-23 16:37:10,657 - INFO - Validated that VN ctest-left_vn-99607138 is not in any agent 2025-07-23 16:37:10,708 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-99607138 info 2025-07-23 16:37:10,708 - INFO - Deleting VN ctest-mgmt_vn-82554464 2025-07-23 16:37:10,878 - DEBUG - Response for deleting network () 2025-07-23 16:37:10,878 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/7b574700-17da-4686-b7f8-940b6fe68421 2025-07-23 16:37:10,887 - DEBUG - Response Code: 404 2025-07-23 16:37:10,887 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:10,894 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:10,956 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:11,057 - INFO - Validated that VN ctest-mgmt_vn-82554464 is not found in API Server 2025-07-23 16:37:11,084 - DEBUG - VN ctest-mgmt_vn-82554464 is not present in Agent 10.0.0.50 2025-07-23 16:37:11,105 - DEBUG - VN ctest-mgmt_vn-82554464 is not present in Agent 10.0.0.130 2025-07-23 16:37:11,105 - INFO - Validated that VN ctest-mgmt_vn-82554464 is not in any agent 2025-07-23 16:37:11,154 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-82554464 info 2025-07-23 16:37:12,481 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 16, in test_svc_hc_basic
    create_svms=True, max_inst=1)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network-nat.qcow2'
2025-07-23 16:37:12,505 - DEBUG - No XMPP flaps were noticed during the test 2025-07-23 16:37:12,505 - INFO - 2025-07-23 16:37:12,505 - INFO - END TEST : test_svc_hc_basic : FAILED[0:01:56] 2025-07-23 16:37:12,505 - INFO - -------------------------------------------------------------------------------- 2025-07-23 16:37:12,508 - INFO - ================================================================================ 2025-07-23 16:37:12,509 - INFO - STARTING TEST : test_svc_trans_segment_right_hc_fail_svm 2025-07-23 16:37:13,860 - DEBUG - Nothing to compare xmpp stats {'10.0.0.50': {'10.20.0.254': '0', '10.20.0.17': '0'}, '10.0.0.130': {'10.20.0.17': '0', '10.20.0.254': '0'}} with 2025-07-23 16:37:13,860 - INFO - Initial checks done. Running the testcase now 2025-07-23 16:37:13,860 - INFO - 2025-07-23 16:37:14,138 - DEBUG - Response for create_network : {'network': {'id': '8ab0edb7-d2b5-4862-ac47-a09d2d90b870', 'name': 'ctest-mgmt_vn-90758724', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-mgmt_vn-90758724'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:37:14,386 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e5f6b099-583d-4670-8bae-1e8f690299dc', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': '8ab0edb7-d2b5-4862-ac47-a09d2d90b870', 'ip_version': 4, 'cidr': '143.48.236.128/26', 'allocation_pools': [{'start': '143.48.236.130', 'end': '143.48.236.190'}], 'gateway_ip': '143.48.236.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '143.48.236.130', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:37:14,415 - INFO - Created VN ctest-mgmt_vn-90758724 2025-07-23 16:37:14,431 - DEBUG - VN ctest-mgmt_vn-90758724 UUID is 8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,506 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:14,513 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:14,525 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:14,627 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,643 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,653 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/08ad7169-d31d-4669-a9d7-407d5bff5e22 2025-07-23 16:37:14,662 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/08ad7169-d31d-4669-a9d7-407d5bff5e22 2025-07-23 16:37:14,677 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/38cac09e-d79d-4499-85d2-ddabfb0d0737 2025-07-23 16:37:14,686 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:37:14,687 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,700 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/08ad7169-d31d-4669-a9d7-407d5bff5e22 2025-07-23 16:37:14,721 - INFO - Verified VN network id 12 for VN 8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,721 - INFO - Verifications in API Server for VN ctest-mgmt_vn-90758724 passed 2025-07-23 16:37:14,721 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,733 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/08ad7169-d31d-4669-a9d7-407d5bff5e22 2025-07-23 16:37:14,742 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/08ad7169-d31d-4669-a9d7-407d5bff5e22 2025-07-23 16:37:14,750 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/38cac09e-d79d-4499-85d2-ddabfb0d0737 2025-07-23 16:37:14,766 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9993748946905024610', 'uuid-lslong': '12414067494804830320'}, 'enable': 'true', 'created': '2025-07-23T16:37:14', 'last-modified': '2025-07-23T16:37:14', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.434112'} 2025-07-23 16:37:14,773 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:37:14,780 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9993748946905024610', 'uuid-lslong': '12414067494804830320'}, 'enable': 'true', 'created': '2025-07-23T16:37:14', 'last-modified': '2025-07-23T16:37:14', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.444549'} 2025-07-23 16:37:14,787 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:37:14,795 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9993748946905024610', 'uuid-lslong': '12414067494804830320'}, 'enable': 'true', 'created': '2025-07-23T16:37:14', 'last-modified': '2025-07-23T16:37:14', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.469292'} 2025-07-23 16:37:14,803 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 16:37:14,803 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-90758724 passed 2025-07-23 16:37:14,804 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-90758724 in API_Server ====== 2025-07-23 16:37:14,804 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:14,812 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:14,827 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:14,930 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/8ab0edb7-d2b5-4862-ac47-a09d2d90b870 2025-07-23 16:37:14,943 - DEBUG - =>VN ctest-mgmt_vn-90758724 has no policy to be verified 2025-07-23 16:37:14,943 - DEBUG - Verifying the vn in opserver 2025-07-23 16:37:14,943 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724 virtual network link through opserver 10.0.0.23 2025-07-23 16:37:14,943 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:37:14,959 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724?flat'} 2025-07-23 16:37:14,959 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724 virtual network link through opserver 10.0.0.249 2025-07-23 16:37:14,959 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:37:14,973 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724?flat'} 2025-07-23 16:37:14,973 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724 virtual network link through opserver 10.0.0.73 2025-07-23 16:37:14,973 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:37:14,989 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724?flat'} 2025-07-23 16:37:14,989 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-mgmt_vn-90758724 is found in opserver 2025-07-23 16:37:15,037 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:37:15,060 - DEBUG - VRF ids for VN ctest-mgmt_vn-90758724: {} 2025-07-23 16:37:15,349 - DEBUG - Response for create_network : {'network': {'id': '252b107a-3c1c-4667-a1b0-3407449ef8a4', 'name': 'ctest-left_vn-60865135', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-left_vn-60865135'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:37:15,602 - DEBUG - Response for create_subnet : {'subnet': {'id': 'cb08b614-73bb-4e9e-a3ed-4f389f2f24b7', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': '252b107a-3c1c-4667-a1b0-3407449ef8a4', 'ip_version': 4, 'cidr': '36.213.144.128/26', 'allocation_pools': [{'start': '36.213.144.130', 'end': '36.213.144.190'}], 'gateway_ip': '36.213.144.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '36.213.144.130', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:37:15,628 - INFO - Created VN ctest-left_vn-60865135 2025-07-23 16:37:15,641 - DEBUG - VN ctest-left_vn-60865135 UUID is 252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:15,720 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:15,728 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:15,741 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:15,844 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:15,859 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:15,870 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/0c23dda0-9797-4ced-8ad0-ce3202b90204 2025-07-23 16:37:15,881 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/0c23dda0-9797-4ced-8ad0-ce3202b90204 2025-07-23 16:37:15,889 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/7922e96d-27c6-4b6f-a0e0-fd00a4650e24 2025-07-23 16:37:15,897 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:37:15,897 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:15,908 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/0c23dda0-9797-4ced-8ad0-ce3202b90204 2025-07-23 16:37:15,926 - INFO - Verified VN network id 13 for VN 252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:15,926 - INFO - Verifications in API Server for VN ctest-left_vn-60865135 passed 2025-07-23 16:37:15,926 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:15,938 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/0c23dda0-9797-4ced-8ad0-ce3202b90204 2025-07-23 16:37:15,948 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/0c23dda0-9797-4ced-8ad0-ce3202b90204 2025-07-23 16:37:15,957 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/7922e96d-27c6-4b6f-a0e0-fd00a4650e24 2025-07-23 16:37:15,970 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2678252520582432359', 'uuid-lslong': '11650869441828157604'}, 'enable': 'true', 'created': '2025-07-23T16:37:15', 'last-modified': '2025-07-23T16:37:15', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.419248'} 2025-07-23 16:37:15,976 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:37:15,983 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2678252520582432359', 'uuid-lslong': '11650869441828157604'}, 'enable': 'true', 'created': '2025-07-23T16:37:15', 'last-modified': '2025-07-23T16:37:15', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.431548'} 2025-07-23 16:37:15,992 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:37:15,999 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2678252520582432359', 'uuid-lslong': '11650869441828157604'}, 'enable': 'true', 'created': '2025-07-23T16:37:15', 'last-modified': '2025-07-23T16:37:15', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.450796'} 2025-07-23 16:37:16,007 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-23 16:37:16,007 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-60865135 passed 2025-07-23 16:37:16,008 - DEBUG - ====Verifying policy data for ctest-left_vn-60865135 in API_Server ====== 2025-07-23 16:37:16,008 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:16,015 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:16,028 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:16,128 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/252b107a-3c1c-4667-a1b0-3407449ef8a4 2025-07-23 16:37:16,140 - DEBUG - =>VN ctest-left_vn-60865135 has no policy to be verified 2025-07-23 16:37:16,141 - DEBUG - Verifying the vn in opserver 2025-07-23 16:37:16,141 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135 virtual network link through opserver 10.0.0.23 2025-07-23 16:37:16,141 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:37:16,161 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135?flat'} 2025-07-23 16:37:16,161 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135 virtual network link through opserver 10.0.0.249 2025-07-23 16:37:16,161 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:37:16,183 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135?flat'} 2025-07-23 16:37:16,183 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135 virtual network link through opserver 10.0.0.73 2025-07-23 16:37:16,183 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:37:16,199 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135?flat'} 2025-07-23 16:37:16,199 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-left_vn-60865135 is found in opserver 2025-07-23 16:37:16,227 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:37:16,242 - DEBUG - VRF ids for VN ctest-left_vn-60865135: {} 2025-07-23 16:37:16,536 - DEBUG - Response for create_network : {'network': {'id': '4db8a482-9002-49af-93d8-d35373838908', 'name': 'ctest-right_vn-07314408', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-right_vn-07314408'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:37:16,795 - DEBUG - Response for create_subnet : {'subnet': {'id': '0bb403e7-eafc-4f58-811e-07d8ab7a73e2', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': '4db8a482-9002-49af-93d8-d35373838908', 'ip_version': 4, 'cidr': '146.48.109.64/26', 'allocation_pools': [{'start': '146.48.109.66', 'end': '146.48.109.126'}], 'gateway_ip': '146.48.109.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '146.48.109.66', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:37:16,818 - INFO - Created VN ctest-right_vn-07314408 2025-07-23 16:37:16,830 - DEBUG - VN ctest-right_vn-07314408 UUID is 4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:16,911 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:16,920 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:16,934 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:17,031 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:17,054 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:17,066 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/d6afc7a1-d2a0-4269-afd6-0039b3b8eb66 2025-07-23 16:37:17,075 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/d6afc7a1-d2a0-4269-afd6-0039b3b8eb66 2025-07-23 16:37:17,083 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/9f613ab8-d764-4935-bd33-351d0974947b 2025-07-23 16:37:17,093 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:37:17,093 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:17,104 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/d6afc7a1-d2a0-4269-afd6-0039b3b8eb66 2025-07-23 16:37:17,125 - INFO - Verified VN network id 14 for VN 4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:17,125 - INFO - Verifications in API Server for VN ctest-right_vn-07314408 passed 2025-07-23 16:37:17,126 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:17,142 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/d6afc7a1-d2a0-4269-afd6-0039b3b8eb66 2025-07-23 16:37:17,157 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/d6afc7a1-d2a0-4269-afd6-0039b3b8eb66 2025-07-23 16:37:17,175 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/9f613ab8-d764-4935-bd33-351d0974947b 2025-07-23 16:37:17,190 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5600407017303984559', 'uuid-lslong': '10653497273918654728'}, 'enable': 'true', 'created': '2025-07-23T16:37:16', 'last-modified': '2025-07-23T16:37:16', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.455970'} 2025-07-23 16:37:17,197 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:37:17,205 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5600407017303984559', 'uuid-lslong': '10653497273918654728'}, 'enable': 'true', 'created': '2025-07-23T16:37:16', 'last-modified': '2025-07-23T16:37:16', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.473100'} 2025-07-23 16:37:17,215 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:37:17,227 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5600407017303984559', 'uuid-lslong': '10653497273918654728'}, 'enable': 'true', 'created': '2025-07-23T16:37:16', 'last-modified': '2025-07-23T16:37:16', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.495589'} 2025-07-23 16:37:17,234 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-23 16:37:17,234 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-07314408 passed 2025-07-23 16:37:17,234 - DEBUG - ====Verifying policy data for ctest-right_vn-07314408 in API_Server ====== 2025-07-23 16:37:17,234 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:17,241 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:17,260 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:17,367 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/4db8a482-9002-49af-93d8-d35373838908 2025-07-23 16:37:17,379 - DEBUG - =>VN ctest-right_vn-07314408 has no policy to be verified 2025-07-23 16:37:17,379 - DEBUG - Verifying the vn in opserver 2025-07-23 16:37:17,379 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408 virtual network link through opserver 10.0.0.23 2025-07-23 16:37:17,379 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:37:17,394 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408?flat'} 2025-07-23 16:37:17,394 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408 virtual network link through opserver 10.0.0.249 2025-07-23 16:37:17,394 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:37:17,410 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408?flat'} 2025-07-23 16:37:17,410 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408 virtual network link through opserver 10.0.0.73 2025-07-23 16:37:17,410 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:37:17,429 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408?flat'} 2025-07-23 16:37:17,429 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-right_vn-07314408 is found in opserver 2025-07-23 16:37:17,466 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:37:17,485 - DEBUG - VRF ids for VN ctest-right_vn-07314408: {} 2025-07-23 16:37:17,798 - DEBUG - Response for create_network : {'network': {'id': '73806c47-15eb-4234-9711-dc9fdb4acaa9', 'name': 'ctest-trans_left_vn-11317619', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-trans_left_vn-11317619'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:37:18,039 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c57aafb8-c3c3-452a-91cc-606e81fa37c0', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': '73806c47-15eb-4234-9711-dc9fdb4acaa9', 'ip_version': 4, 'cidr': '55.201.28.64/26', 'allocation_pools': [{'start': '55.201.28.66', 'end': '55.201.28.126'}], 'gateway_ip': '55.201.28.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '55.201.28.66', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:37:18,068 - INFO - Created VN ctest-trans_left_vn-11317619 2025-07-23 16:37:18,088 - DEBUG - VN ctest-trans_left_vn-11317619 UUID is 73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,280 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:18,289 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:18,308 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:18,425 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,440 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,456 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/99eecfc1-de93-4b37-bf3e-0a618ff368a6 2025-07-23 16:37:18,469 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/99eecfc1-de93-4b37-bf3e-0a618ff368a6 2025-07-23 16:37:18,481 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/a0f2e56c-20c8-4cce-b704-445d411b5c59 2025-07-23 16:37:18,493 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-23 16:37:18,494 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,513 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/99eecfc1-de93-4b37-bf3e-0a618ff368a6 2025-07-23 16:37:18,534 - INFO - Verified VN network id 15 for VN 73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,534 - INFO - Verifications in API Server for VN ctest-trans_left_vn-11317619 passed 2025-07-23 16:37:18,534 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,553 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/99eecfc1-de93-4b37-bf3e-0a618ff368a6 2025-07-23 16:37:18,564 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/99eecfc1-de93-4b37-bf3e-0a618ff368a6 2025-07-23 16:37:18,577 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/a0f2e56c-20c8-4cce-b704-445d411b5c59 2025-07-23 16:37:18,596 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8322771163946893876', 'uuid-lslong': '10885724353468222121'}, 'enable': 'true', 'created': '2025-07-23T16:37:17', 'last-modified': '2025-07-23T16:37:17', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.623470'} 2025-07-23 16:37:18,605 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-23 16:37:18,612 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8322771163946893876', 'uuid-lslong': '10885724353468222121'}, 'enable': 'true', 'created': '2025-07-23T16:37:17', 'last-modified': '2025-07-23T16:37:17', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.641275'} 2025-07-23 16:37:18,622 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-23 16:37:18,631 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8322771163946893876', 'uuid-lslong': '10885724353468222121'}, 'enable': 'true', 'created': '2025-07-23T16:37:17', 'last-modified': '2025-07-23T16:37:17', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.661705'} 2025-07-23 16:37:18,644 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-23 16:37:18,645 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_left_vn-11317619 passed 2025-07-23 16:37:18,645 - DEBUG - ====Verifying policy data for ctest-trans_left_vn-11317619 in API_Server ====== 2025-07-23 16:37:18,645 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:18,655 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:18,672 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:18,782 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/73806c47-15eb-4234-9711-dc9fdb4acaa9 2025-07-23 16:37:18,793 - DEBUG - =>VN ctest-trans_left_vn-11317619 has no policy to be verified 2025-07-23 16:37:18,793 - DEBUG - Verifying the vn in opserver 2025-07-23 16:37:18,793 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619 virtual network link through opserver 10.0.0.23 2025-07-23 16:37:18,793 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:37:18,809 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619?flat'} 2025-07-23 16:37:18,809 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619 virtual network link through opserver 10.0.0.249 2025-07-23 16:37:18,810 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:37:18,827 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619?flat'} 2025-07-23 16:37:18,827 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619 virtual network link through opserver 10.0.0.73 2025-07-23 16:37:18,827 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:37:18,843 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619?flat'} 2025-07-23 16:37:18,843 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-trans_left_vn-11317619 is found in opserver 2025-07-23 16:37:18,886 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:37:18,910 - DEBUG - VRF ids for VN ctest-trans_left_vn-11317619: {} 2025-07-23 16:37:19,201 - DEBUG - Response for create_network : {'network': {'id': 'f46b4021-a552-4ace-9817-76bb9ca5c26c', 'name': 'ctest-trans_right_vn-31187517', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'project_id': '408ab2c6db684e71b35dec9e479780ab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-76392771', 'ctest-trans_right_vn-31187517'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 16:37:19,462 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e5a5faa5-d3f9-418a-a8e9-fc13883d2af4', 'name': '', 'tenant_id': '408ab2c6db684e71b35dec9e479780ab', 'network_id': 'f46b4021-a552-4ace-9817-76bb9ca5c26c', 'ip_version': 4, 'cidr': '152.202.217.0/26', 'allocation_pools': [{'start': '152.202.217.2', 'end': '152.202.217.62'}], 'gateway_ip': '152.202.217.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '152.202.217.2', 'tags': [], 'project_id': '408ab2c6db684e71b35dec9e479780ab'}} 2025-07-23 16:37:19,486 - INFO - Created VN ctest-trans_right_vn-31187517 2025-07-23 16:37:19,498 - DEBUG - VN ctest-trans_right_vn-31187517 UUID is f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:19,585 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:19,593 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:19,607 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:19,707 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:19,726 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:19,739 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/66a2dcdd-82c6-4030-b690-270a2b73caaf 2025-07-23 16:37:19,754 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/66a2dcdd-82c6-4030-b690-270a2b73caaf 2025-07-23 16:37:19,762 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/fa38636a-d750-4868-85b0-e5224300bf0c 2025-07-23 16:37:19,771 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-23 16:37:19,771 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:19,784 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/66a2dcdd-82c6-4030-b690-270a2b73caaf 2025-07-23 16:37:19,803 - INFO - Verified VN network id 16 for VN f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:19,804 - INFO - Verifications in API Server for VN ctest-trans_right_vn-31187517 passed 2025-07-23 16:37:19,804 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:19,815 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/66a2dcdd-82c6-4030-b690-270a2b73caaf 2025-07-23 16:37:19,824 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/66a2dcdd-82c6-4030-b690-270a2b73caaf 2025-07-23 16:37:19,833 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/fa38636a-d750-4868-85b0-e5224300bf0c 2025-07-23 16:37:19,846 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17612241281014188750', 'uuid-lslong': '10959358766388462188'}, 'enable': 'true', 'created': '2025-07-23T16:37:19', 'last-modified': '2025-07-23T16:37:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.443925'} 2025-07-23 16:37:19,854 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-23 16:37:19,860 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17612241281014188750', 'uuid-lslong': '10959358766388462188'}, 'enable': 'true', 'created': '2025-07-23T16:37:19', 'last-modified': '2025-07-23T16:37:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.458173'} 2025-07-23 16:37:19,868 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-23 16:37:19,873 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-76392771', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17612241281014188750', 'uuid-lslong': '10959358766388462188'}, 'enable': 'true', 'created': '2025-07-23T16:37:19', 'last-modified': '2025-07-23T16:37:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.472076'} 2025-07-23 16:37:19,881 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-23 16:37:19,881 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_right_vn-31187517 passed 2025-07-23 16:37:19,882 - DEBUG - ====Verifying policy data for ctest-trans_right_vn-31187517 in API_Server ====== 2025-07-23 16:37:19,882 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:37:19,890 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:37:19,903 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:37:19,998 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/f46b4021-a552-4ace-9817-76bb9ca5c26c 2025-07-23 16:37:20,012 - DEBUG - =>VN ctest-trans_right_vn-31187517 has no policy to be verified 2025-07-23 16:37:20,012 - DEBUG - Verifying the vn in opserver 2025-07-23 16:37:20,012 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517 virtual network link through opserver 10.0.0.23 2025-07-23 16:37:20,012 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 16:37:20,025 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517?flat'} 2025-07-23 16:37:20,025 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517 virtual network link through opserver 10.0.0.249 2025-07-23 16:37:20,025 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 16:37:20,041 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517?flat'} 2025-07-23 16:37:20,041 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517 virtual network link through opserver 10.0.0.73 2025-07-23 16:37:20,041 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 16:37:20,055 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517?flat'} 2025-07-23 16:37:20,055 - INFO - Validated that VN default-domain:ctest-TestSvcHC-76392771:ctest-trans_right_vn-31187517 is found in opserver 2025-07-23 16:37:20,094 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 16:37:20,110 - DEBUG - VRF ids for VN ctest-trans_right_vn-31187517: {} 2025-07-23 16:37:21,256 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3704-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3704-1) 2025-07-23 16:37:22,807 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3704-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3704-2) 2025-07-23 16:37:22,808 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-05452351'] 2025-07-23 16:37:23,194 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-05452351'] created succesfully 2025-07-23 16:39:03,891 - DEBUG - Installing image tiny_trans_fw 2025-07-23 16:39:03,891 - DEBUG - Image info {'name': 'tinycore-net-transparent-fw.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-07-23 16:39:03,892 - DEBUG - Image build path file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-23 16:39:03,892 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-23 16:39:03,892 - DEBUG - Unzip image 2025-07-23 16:39:03,892 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-23 16:39:03,913 - DEBUG - Try to create image in user context 2025-07-23 16:39:03,976 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-05452351'] 2025-07-23 16:39:04,071 - INFO - Service template: ['default-domain', 'ctest-service_template_1-05452351'] deleted successfully. 2025-07-23 16:39:04,071 - INFO - Skip interface_detach for VM ctest-right_vm-66788214 in state BUILD 2025-07-23 16:39:04,072 - INFO - Deleting VM ctest-right_vm-66788214 2025-07-23 16:39:04,072 - ERROR - VM ctest-right_vm-66788214 has failed to come up 2025-07-23 16:39:04,072 - ERROR - Nova failed to get host of the VM 2025-07-23 16:39:04,072 - ERROR - VM ctest-right_vm-66788214 has failed to come up 2025-07-23 16:39:04,072 - ERROR - Nova failed to get host of the VM 2025-07-23 16:39:04,144 - INFO - Skip interface_detach for VM ctest-left_vm-88113857 in state BUILD 2025-07-23 16:39:04,144 - INFO - Deleting VM ctest-left_vm-88113857 2025-07-23 16:39:04,145 - ERROR - VM ctest-left_vm-88113857 has failed to come up 2025-07-23 16:39:04,145 - ERROR - Nova failed to get host of the VM 2025-07-23 16:39:04,145 - ERROR - VM ctest-left_vm-88113857 has failed to come up 2025-07-23 16:39:04,145 - ERROR - Nova failed to get host of the VM 2025-07-23 16:39:04,213 - INFO - Deleting VN ctest-trans_right_vn-31187517 2025-07-23 16:39:04,382 - DEBUG - Response for deleting network () 2025-07-23 16:39:04,382 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/66a2dcdd-82c6-4030-b690-270a2b73caaf 2025-07-23 16:39:04,389 - DEBUG - Response Code: 404 2025-07-23 16:39:04,390 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:39:04,397 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:39:04,413 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:39:04,522 - INFO - Validated that VN ctest-trans_right_vn-31187517 is not found in API Server 2025-07-23 16:39:04,554 - DEBUG - VN ctest-trans_right_vn-31187517 is not present in Agent 10.0.0.50 2025-07-23 16:39:04,567 - DEBUG - VN ctest-trans_right_vn-31187517 is not present in Agent 10.0.0.130 2025-07-23 16:39:04,567 - INFO - Validated that VN ctest-trans_right_vn-31187517 is not in any agent 2025-07-23 16:39:04,623 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_right_vn-31187517 info 2025-07-23 16:39:04,623 - INFO - Deleting VN ctest-trans_left_vn-11317619 2025-07-23 16:39:04,791 - DEBUG - Response for deleting network () 2025-07-23 16:39:04,791 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/99eecfc1-de93-4b37-bf3e-0a618ff368a6 2025-07-23 16:39:04,817 - DEBUG - Response Code: 404 2025-07-23 16:39:04,818 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:39:04,828 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:39:04,845 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:39:04,965 - INFO - Validated that VN ctest-trans_left_vn-11317619 is not found in API Server 2025-07-23 16:39:04,980 - DEBUG - VN ctest-trans_left_vn-11317619 is not present in Agent 10.0.0.50 2025-07-23 16:39:04,996 - DEBUG - VN ctest-trans_left_vn-11317619 is not present in Agent 10.0.0.130 2025-07-23 16:39:04,996 - INFO - Validated that VN ctest-trans_left_vn-11317619 is not in any agent 2025-07-23 16:39:05,056 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_left_vn-11317619 info 2025-07-23 16:39:05,057 - INFO - Deleting VN ctest-right_vn-07314408 2025-07-23 16:39:05,261 - DEBUG - Response for deleting network () 2025-07-23 16:39:05,261 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/d6afc7a1-d2a0-4269-afd6-0039b3b8eb66 2025-07-23 16:39:05,268 - DEBUG - Response Code: 404 2025-07-23 16:39:05,268 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:39:05,293 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:39:05,334 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:39:05,427 - INFO - Validated that VN ctest-right_vn-07314408 is not found in API Server 2025-07-23 16:39:05,461 - DEBUG - VN ctest-right_vn-07314408 is not present in Agent 10.0.0.50 2025-07-23 16:39:05,479 - DEBUG - VN ctest-right_vn-07314408 is not present in Agent 10.0.0.130 2025-07-23 16:39:05,479 - INFO - Validated that VN ctest-right_vn-07314408 is not in any agent 2025-07-23 16:39:05,548 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-07314408 info 2025-07-23 16:39:05,549 - INFO - Deleting VN ctest-left_vn-60865135 2025-07-23 16:39:05,728 - DEBUG - Response for deleting network () 2025-07-23 16:39:05,729 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/0c23dda0-9797-4ced-8ad0-ce3202b90204 2025-07-23 16:39:05,735 - DEBUG - Response Code: 404 2025-07-23 16:39:05,735 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:39:05,756 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:39:05,775 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:39:05,880 - INFO - Validated that VN ctest-left_vn-60865135 is not found in API Server 2025-07-23 16:39:05,911 - DEBUG - VN ctest-left_vn-60865135 is not present in Agent 10.0.0.50 2025-07-23 16:39:05,926 - DEBUG - VN ctest-left_vn-60865135 is not present in Agent 10.0.0.130 2025-07-23 16:39:05,926 - INFO - Validated that VN ctest-left_vn-60865135 is not in any agent 2025-07-23 16:39:05,980 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-60865135 info 2025-07-23 16:39:05,980 - INFO - Deleting VN ctest-mgmt_vn-90758724 2025-07-23 16:39:06,194 - DEBUG - Response for deleting network () 2025-07-23 16:39:06,194 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/08ad7169-d31d-4669-a9d7-407d5bff5e22 2025-07-23 16:39:06,201 - DEBUG - Response Code: 404 2025-07-23 16:39:06,201 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 16:39:06,208 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 16:39:06,272 - DEBUG - Requesting: http://10.0.0.23:8082/project/408ab2c6-db68-4e71-b35d-ec9e479780ab 2025-07-23 16:39:06,386 - INFO - Validated that VN ctest-mgmt_vn-90758724 is not found in API Server 2025-07-23 16:39:06,420 - DEBUG - VN ctest-mgmt_vn-90758724 is not present in Agent 10.0.0.50 2025-07-23 16:39:06,432 - DEBUG - VN ctest-mgmt_vn-90758724 is not present in Agent 10.0.0.130 2025-07-23 16:39:06,432 - INFO - Validated that VN ctest-mgmt_vn-90758724 is not in any agent 2025-07-23 16:39:06,492 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-90758724 info 2025-07-23 16:39:07,833 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 137, in test_svc_trans_segment_right_hc_fail_svm
    assert self.svc_hc_fail_svm_common()
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 180, in svc_hc_fail_svm_common
    create_svms=True, max_inst=1)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-net-transparent-fw.qcow2'
2025-07-23 16:39:07,852 - DEBUG - No XMPP flaps were noticed during the test 2025-07-23 16:39:07,852 - INFO - 2025-07-23 16:39:07,852 - INFO - END TEST : test_svc_trans_segment_right_hc_fail_svm : FAILED[0:01:55] 2025-07-23 16:39:07,853 - INFO - -------------------------------------------------------------------------------- 2025-07-23 16:39:08,609 - INFO - Deleted project: ctest-TestSvcHC-76392771, ID : 408ab2c6-db68-4e71-b35d-ec9e479780ab