2025-07-24 04:17:31,882 - INFO - Domain Default found not creating 2025-07-24 04:17:32,071 - INFO - Project ctest-TestSvcHC-09960119 not found, creating it 2025-07-24 04:17:32,685 - INFO - Created Project:ctest-TestSvcHC-09960119, ID : 03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:34,313 - INFO - ================================================================================ 2025-07-24 04:17:34,313 - INFO - STARTING TEST : test_svc_hc_basic 2025-07-24 04:17:35,612 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 04:17:35,612 - INFO - Initial checks done. Running the testcase now 2025-07-24 04:17:35,613 - INFO - 2025-07-24 04:17:36,345 - DEBUG - Response for create_network : {'network': {'id': 'a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93', 'name': 'ctest-mgmt_vn-99297399', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-mgmt_vn-99297399'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:17:36,584 - DEBUG - Response for create_subnet : {'subnet': {'id': 'ac63f9d5-6277-4d80-8671-c6b010e22d19', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': 'a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93', 'ip_version': 4, 'cidr': '85.46.255.192/26', 'allocation_pools': [{'start': '85.46.255.194', 'end': '85.46.255.254'}], 'gateway_ip': '85.46.255.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '85.46.255.194', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:17:36,609 - INFO - Created VN ctest-mgmt_vn-99297399 2025-07-24 04:17:36,624 - DEBUG - VN ctest-mgmt_vn-99297399 UUID is a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:36,708 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:37,016 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:37,049 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:17:37,072 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:37,184 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:37,201 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:37,212 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/4a9bfc2b-03cd-4e13-a5b5-907c6932ede0 2025-07-24 04:17:37,221 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/4a9bfc2b-03cd-4e13-a5b5-907c6932ede0 2025-07-24 04:17:37,229 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/7c89bdf9-6f9e-4598-ab8f-0d63a2cbc507 2025-07-24 04:17:37,236 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:17:37,236 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:37,249 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/4a9bfc2b-03cd-4e13-a5b5-907c6932ede0 2025-07-24 04:17:37,274 - INFO - Verified VN network id 11 for VN a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:37,275 - INFO - Verifications in API Server for VN ctest-mgmt_vn-99297399 passed 2025-07-24 04:17:37,275 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:37,288 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/4a9bfc2b-03cd-4e13-a5b5-907c6932ede0 2025-07-24 04:17:37,298 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/4a9bfc2b-03cd-4e13-a5b5-907c6932ede0 2025-07-24 04:17:37,309 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/7c89bdf9-6f9e-4598-ab8f-0d63a2cbc507 2025-07-24 04:17:37,329 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11942397363058855186', 'uuid-lslong': '10929855458503961491'}, 'enable': 'true', 'created': '2025-07-24T04:17:36', 'last-modified': '2025-07-24T04:17:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.805212'} 2025-07-24 04:17:37,338 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:17:37,345 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11942397363058855186', 'uuid-lslong': '10929855458503961491'}, 'enable': 'true', 'created': '2025-07-24T04:17:36', 'last-modified': '2025-07-24T04:17:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.821565'} 2025-07-24 04:17:37,353 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:17:37,361 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11942397363058855186', 'uuid-lslong': '10929855458503961491'}, 'enable': 'true', 'created': '2025-07-24T04:17:36', 'last-modified': '2025-07-24T04:17:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.838203'} 2025-07-24 04:17:37,369 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:17:37,369 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-99297399 passed 2025-07-24 04:17:37,369 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-99297399 in API_Server ====== 2025-07-24 04:17:37,369 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:37,377 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:17:37,391 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:37,497 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/a5bbeb20-cfaf-4d12-97ae-a5a1727e2b93 2025-07-24 04:17:37,509 - DEBUG - =>VN ctest-mgmt_vn-99297399 has no policy to be verified 2025-07-24 04:17:37,509 - DEBUG - Verifying the vn in opserver 2025-07-24 04:17:37,509 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399 virtual network link through opserver 10.0.0.19 2025-07-24 04:17:37,509 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:17:37,814 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:17:37,882 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399?flat'} 2025-07-24 04:17:37,882 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399 virtual network link through opserver 10.0.0.38 2025-07-24 04:17:37,882 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:17:38,201 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:17:38,264 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399?flat'} 2025-07-24 04:17:38,264 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399 virtual network link through opserver 10.0.0.41 2025-07-24 04:17:38,264 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:17:38,578 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:17:38,667 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399?flat'} 2025-07-24 04:17:38,667 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-99297399 is found in opserver 2025-07-24 04:17:38,697 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:17:38,710 - DEBUG - VRF ids for VN ctest-mgmt_vn-99297399: {} 2025-07-24 04:17:39,008 - DEBUG - Response for create_network : {'network': {'id': 'cfed6b56-1cef-4d45-a773-c01d99dfa48b', 'name': 'ctest-left_vn-69820815', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-left_vn-69820815'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:17:39,307 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c6aff505-ccec-4382-b102-7c4bf54d3b29', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': 'cfed6b56-1cef-4d45-a773-c01d99dfa48b', 'ip_version': 4, 'cidr': '106.167.108.64/26', 'allocation_pools': [{'start': '106.167.108.66', 'end': '106.167.108.126'}], 'gateway_ip': '106.167.108.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '106.167.108.66', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:17:39,333 - INFO - Created VN ctest-left_vn-69820815 2025-07-24 04:17:39,349 - DEBUG - VN ctest-left_vn-69820815 UUID is cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,433 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:39,442 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:17:39,460 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:39,567 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,581 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,593 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/28b783ee-6f8f-4269-b968-f27081d7ab41 2025-07-24 04:17:39,606 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/28b783ee-6f8f-4269-b968-f27081d7ab41 2025-07-24 04:17:39,616 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/382fa575-2029-4b8e-8582-d76a9bff0899 2025-07-24 04:17:39,626 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:17:39,626 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,637 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/28b783ee-6f8f-4269-b968-f27081d7ab41 2025-07-24 04:17:39,659 - INFO - Verified VN network id 12 for VN cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,659 - INFO - Verifications in API Server for VN ctest-left_vn-69820815 passed 2025-07-24 04:17:39,660 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,673 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/28b783ee-6f8f-4269-b968-f27081d7ab41 2025-07-24 04:17:39,683 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/28b783ee-6f8f-4269-b968-f27081d7ab41 2025-07-24 04:17:39,694 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/382fa575-2029-4b8e-8582-d76a9bff0899 2025-07-24 04:17:39,711 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14982749552928312645', 'uuid-lslong': '12066199060023846027'}, 'enable': 'true', 'created': '2025-07-24T04:17:38', 'last-modified': '2025-07-24T04:17:39', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.481925'} 2025-07-24 04:17:39,719 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:17:39,726 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14982749552928312645', 'uuid-lslong': '12066199060023846027'}, 'enable': 'true', 'created': '2025-07-24T04:17:38', 'last-modified': '2025-07-24T04:17:39', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.503136'} 2025-07-24 04:17:39,734 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:17:39,741 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14982749552928312645', 'uuid-lslong': '12066199060023846027'}, 'enable': 'true', 'created': '2025-07-24T04:17:38', 'last-modified': '2025-07-24T04:17:39', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.517190'} 2025-07-24 04:17:39,749 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:17:39,750 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-69820815 passed 2025-07-24 04:17:39,750 - DEBUG - ====Verifying policy data for ctest-left_vn-69820815 in API_Server ====== 2025-07-24 04:17:39,750 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:39,758 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:17:39,774 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:39,882 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/cfed6b56-1cef-4d45-a773-c01d99dfa48b 2025-07-24 04:17:39,895 - DEBUG - =>VN ctest-left_vn-69820815 has no policy to be verified 2025-07-24 04:17:39,895 - DEBUG - Verifying the vn in opserver 2025-07-24 04:17:39,895 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815 virtual network link through opserver 10.0.0.19 2025-07-24 04:17:39,895 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:17:39,962 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815?flat'} 2025-07-24 04:17:39,962 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815 virtual network link through opserver 10.0.0.38 2025-07-24 04:17:39,962 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:17:40,027 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815?flat'} 2025-07-24 04:17:40,027 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815 virtual network link through opserver 10.0.0.41 2025-07-24 04:17:40,027 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:17:40,095 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815?flat'} 2025-07-24 04:17:40,095 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-69820815 is found in opserver 2025-07-24 04:17:40,128 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:17:40,142 - DEBUG - VRF ids for VN ctest-left_vn-69820815: {} 2025-07-24 04:17:40,468 - DEBUG - Response for create_network : {'network': {'id': '4e6211c2-155c-4715-a039-87a527a1f4f4', 'name': 'ctest-right_vn-94235058', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-right_vn-94235058'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:17:40,789 - DEBUG - Response for create_subnet : {'subnet': {'id': '6550a1e4-8c56-4058-b001-ba3d524d89b7', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': '4e6211c2-155c-4715-a039-87a527a1f4f4', 'ip_version': 4, 'cidr': '30.201.2.192/26', 'allocation_pools': [{'start': '30.201.2.194', 'end': '30.201.2.254'}], 'gateway_ip': '30.201.2.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '30.201.2.194', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:17:40,813 - INFO - Created VN ctest-right_vn-94235058 2025-07-24 04:17:40,834 - DEBUG - VN ctest-right_vn-94235058 UUID is 4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:40,933 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:40,942 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:17:40,962 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:41,067 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:41,082 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:41,096 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/51deefbf-cb9a-4b17-a112-887347a7c0f4 2025-07-24 04:17:41,107 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/51deefbf-cb9a-4b17-a112-887347a7c0f4 2025-07-24 04:17:41,116 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/1de4a423-3c6a-40be-bd8d-ecc54c7e5dc5 2025-07-24 04:17:41,124 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:17:41,124 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:41,136 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/51deefbf-cb9a-4b17-a112-887347a7c0f4 2025-07-24 04:17:41,156 - INFO - Verified VN network id 13 for VN 4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:41,156 - INFO - Verifications in API Server for VN ctest-right_vn-94235058 passed 2025-07-24 04:17:41,156 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:41,168 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/51deefbf-cb9a-4b17-a112-887347a7c0f4 2025-07-24 04:17:41,176 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/51deefbf-cb9a-4b17-a112-887347a7c0f4 2025-07-24 04:17:41,186 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/1de4a423-3c6a-40be-bd8d-ecc54c7e5dc5 2025-07-24 04:17:41,202 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5648096407955719957', 'uuid-lslong': '11545408263145256180'}, 'enable': 'true', 'created': '2025-07-24T04:17:40', 'last-modified': '2025-07-24T04:17:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.496037'} 2025-07-24 04:17:41,213 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:17:41,223 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5648096407955719957', 'uuid-lslong': '11545408263145256180'}, 'enable': 'true', 'created': '2025-07-24T04:17:40', 'last-modified': '2025-07-24T04:17:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.516520'} 2025-07-24 04:17:41,231 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:17:41,239 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5648096407955719957', 'uuid-lslong': '11545408263145256180'}, 'enable': 'true', 'created': '2025-07-24T04:17:40', 'last-modified': '2025-07-24T04:17:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.529656'} 2025-07-24 04:17:41,256 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:17:41,256 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-94235058 passed 2025-07-24 04:17:41,256 - DEBUG - ====Verifying policy data for ctest-right_vn-94235058 in API_Server ====== 2025-07-24 04:17:41,256 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:17:41,263 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:17:41,279 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:17:41,389 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4e6211c2-155c-4715-a039-87a527a1f4f4 2025-07-24 04:17:41,406 - DEBUG - =>VN ctest-right_vn-94235058 has no policy to be verified 2025-07-24 04:17:41,406 - DEBUG - Verifying the vn in opserver 2025-07-24 04:17:41,406 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058 virtual network link through opserver 10.0.0.19 2025-07-24 04:17:41,406 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:17:41,475 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058?flat'} 2025-07-24 04:17:41,475 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058 virtual network link through opserver 10.0.0.38 2025-07-24 04:17:41,475 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:17:41,546 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058?flat'} 2025-07-24 04:17:41,546 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058 virtual network link through opserver 10.0.0.41 2025-07-24 04:17:41,546 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:17:41,620 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058?flat'} 2025-07-24 04:17:41,620 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-94235058 is found in opserver 2025-07-24 04:17:41,650 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:17:41,667 - DEBUG - VRF ids for VN ctest-right_vn-94235058: {} 2025-07-24 04:17:41,738 - DEBUG - Services list from nova: [, , , ] 2025-07-24 04:17:42,994 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3712-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3712-1) 2025-07-24 04:17:45,101 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3712-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3712-2) 2025-07-24 04:17:45,101 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-87920705'] 2025-07-24 04:17:45,429 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-87920705'] created succesfully 2025-07-24 04:19:26,337 - DEBUG - Installing image tiny_nat_fw 2025-07-24 04:19:26,337 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vctmpl': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmtx', 'vcname': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmdk', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-07-24 04:19:26,338 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-24 04:19:26,338 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-24 04:19:26,338 - DEBUG - Unzip image 2025-07-24 04:19:26,338 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-24 04:19:26,362 - DEBUG - Try to create image in user context 2025-07-24 04:19:26,430 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-87920705'] 2025-07-24 04:19:26,544 - INFO - Service template: ['default-domain', 'ctest-service_template_1-87920705'] deleted successfully. 2025-07-24 04:19:26,544 - INFO - Skip interface_detach for VM ctest-right_vm-21164221 in state BUILD 2025-07-24 04:19:26,544 - INFO - Deleting VM ctest-right_vm-21164221 2025-07-24 04:19:26,577 - ERROR - VM ctest-right_vm-21164221 has failed to come up 2025-07-24 04:19:26,577 - ERROR - Nova failed to get host of the VM 2025-07-24 04:19:26,578 - ERROR - VM ctest-right_vm-21164221 has failed to come up 2025-07-24 04:19:26,578 - ERROR - Nova failed to get host of the VM 2025-07-24 04:19:26,649 - INFO - Skip interface_detach for VM ctest-left_vm-28197314 in state BUILD 2025-07-24 04:19:26,649 - INFO - Deleting VM ctest-left_vm-28197314 2025-07-24 04:19:26,649 - ERROR - VM ctest-left_vm-28197314 has failed to come up 2025-07-24 04:19:26,649 - ERROR - Nova failed to get host of the VM 2025-07-24 04:19:26,650 - ERROR - VM ctest-left_vm-28197314 has failed to come up 2025-07-24 04:19:26,650 - ERROR - Nova failed to get host of the VM 2025-07-24 04:19:26,725 - INFO - Deleting VN ctest-right_vn-94235058 2025-07-24 04:19:26,775 - DEBUG - VN 4e6211c2-155c-4715-a039-87a527a1f4f4 still in use: Unable to complete operation on network 4e6211c2-155c-4715-a039-87a527a1f4f4. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-ae0b16e9-be5b-48aa-b4a7-d29490213557'] 2025-07-24 04:19:26,775 - WARNING - Deleting VN ctest-right_vn-94235058 failed..Will retry 2025-07-24 04:19:28,982 - DEBUG - Response for deleting network () 2025-07-24 04:19:28,982 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/51deefbf-cb9a-4b17-a112-887347a7c0f4 2025-07-24 04:19:28,990 - DEBUG - Response Code: 404 2025-07-24 04:19:28,990 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:28,998 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:29,077 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:29,182 - INFO - Validated that VN ctest-right_vn-94235058 is not found in API Server 2025-07-24 04:19:29,207 - DEBUG - VN ctest-right_vn-94235058 is not present in Agent 10.0.0.57 2025-07-24 04:19:29,221 - DEBUG - VN ctest-right_vn-94235058 is not present in Agent 10.0.0.23 2025-07-24 04:19:29,221 - INFO - Validated that VN ctest-right_vn-94235058 is not in any agent 2025-07-24 04:19:29,270 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-94235058 info 2025-07-24 04:19:29,270 - INFO - Deleting VN ctest-left_vn-69820815 2025-07-24 04:19:29,456 - DEBUG - Response for deleting network () 2025-07-24 04:19:29,456 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/28b783ee-6f8f-4269-b968-f27081d7ab41 2025-07-24 04:19:29,466 - DEBUG - Response Code: 404 2025-07-24 04:19:29,466 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:29,475 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:29,535 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:29,634 - INFO - Validated that VN ctest-left_vn-69820815 is not found in API Server 2025-07-24 04:19:29,649 - DEBUG - VN ctest-left_vn-69820815 is not present in Agent 10.0.0.57 2025-07-24 04:19:29,665 - DEBUG - VN ctest-left_vn-69820815 is not present in Agent 10.0.0.23 2025-07-24 04:19:29,665 - INFO - Validated that VN ctest-left_vn-69820815 is not in any agent 2025-07-24 04:19:29,715 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-69820815 info 2025-07-24 04:19:29,716 - INFO - Deleting VN ctest-mgmt_vn-99297399 2025-07-24 04:19:29,969 - DEBUG - Response for deleting network () 2025-07-24 04:19:29,969 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/4a9bfc2b-03cd-4e13-a5b5-907c6932ede0 2025-07-24 04:19:29,977 - DEBUG - Response Code: 404 2025-07-24 04:19:29,978 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:29,986 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:30,001 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:30,101 - INFO - Validated that VN ctest-mgmt_vn-99297399 is not found in API Server 2025-07-24 04:19:30,120 - DEBUG - VN ctest-mgmt_vn-99297399 is not present in Agent 10.0.0.57 2025-07-24 04:19:30,136 - DEBUG - VN ctest-mgmt_vn-99297399 is not present in Agent 10.0.0.23 2025-07-24 04:19:30,136 - INFO - Validated that VN ctest-mgmt_vn-99297399 is not in any agent 2025-07-24 04:19:30,184 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-99297399 info 2025-07-24 04:19:31,479 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 16, in test_svc_hc_basic
    create_svms=True, max_inst=1)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network-nat.qcow2'
2025-07-24 04:19:31,500 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 04:19:31,500 - INFO -
2025-07-24 04:19:31,500 - INFO - END TEST : test_svc_hc_basic : FAILED[0:01:57]
2025-07-24 04:19:31,500 - INFO - -------------------------------------------------------------------------------- 2025-07-24 04:19:31,504 - INFO - ================================================================================ 2025-07-24 04:19:31,504 - INFO - STARTING TEST : test_svc_trans_segment_right_hc_fail_svm 2025-07-24 04:19:32,811 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 04:19:32,811 - INFO - Initial checks done. Running the testcase now 2025-07-24 04:19:32,811 - INFO - 2025-07-24 04:19:33,126 - DEBUG - Response for create_network : {'network': {'id': '6f280f0d-c56d-4893-b814-e40d0c889cb3', 'name': 'ctest-mgmt_vn-80255049', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-mgmt_vn-80255049'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:19:33,364 - DEBUG - Response for create_subnet : {'subnet': {'id': '44697f3d-fd76-4cb8-a8ea-19feba350e2e', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': '6f280f0d-c56d-4893-b814-e40d0c889cb3', 'ip_version': 4, 'cidr': '153.205.90.128/26', 'allocation_pools': [{'start': '153.205.90.130', 'end': '153.205.90.190'}], 'gateway_ip': '153.205.90.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '153.205.90.130', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:19:33,391 - INFO - Created VN ctest-mgmt_vn-80255049 2025-07-24 04:19:33,407 - DEBUG - VN ctest-mgmt_vn-80255049 UUID is 6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,499 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:33,508 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:33,522 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:33,632 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,647 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,664 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/40645a59-d120-4235-b1f3-504150d94f34 2025-07-24 04:19:33,676 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/40645a59-d120-4235-b1f3-504150d94f34 2025-07-24 04:19:33,687 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/b8f4c2c0-d6eb-4eaf-914c-9d33ec56bb5c 2025-07-24 04:19:33,696 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:19:33,696 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,708 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/40645a59-d120-4235-b1f3-504150d94f34 2025-07-24 04:19:33,731 - INFO - Verified VN network id 11 for VN 6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,731 - INFO - Verifications in API Server for VN ctest-mgmt_vn-80255049 passed 2025-07-24 04:19:33,731 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,744 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/40645a59-d120-4235-b1f3-504150d94f34 2025-07-24 04:19:33,754 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/40645a59-d120-4235-b1f3-504150d94f34 2025-07-24 04:19:33,764 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/b8f4c2c0-d6eb-4eaf-914c-9d33ec56bb5c 2025-07-24 04:19:33,780 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8009668489099692179', 'uuid-lslong': '13264477547208940723'}, 'enable': 'true', 'created': '2025-07-24T04:19:33', 'last-modified': '2025-07-24T04:19:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.479533'} 2025-07-24 04:19:33,790 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:19:33,797 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8009668489099692179', 'uuid-lslong': '13264477547208940723'}, 'enable': 'true', 'created': '2025-07-24T04:19:33', 'last-modified': '2025-07-24T04:19:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.502544'} 2025-07-24 04:19:33,804 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:19:33,812 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8009668489099692179', 'uuid-lslong': '13264477547208940723'}, 'enable': 'true', 'created': '2025-07-24T04:19:33', 'last-modified': '2025-07-24T04:19:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.516863'} 2025-07-24 04:19:33,821 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-24 04:19:33,821 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-80255049 passed 2025-07-24 04:19:33,821 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-80255049 in API_Server ====== 2025-07-24 04:19:33,821 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:33,829 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:33,849 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:33,959 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6f280f0d-c56d-4893-b814-e40d0c889cb3 2025-07-24 04:19:33,974 - DEBUG - =>VN ctest-mgmt_vn-80255049 has no policy to be verified 2025-07-24 04:19:33,975 - DEBUG - Verifying the vn in opserver 2025-07-24 04:19:33,975 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049 virtual network link through opserver 10.0.0.19 2025-07-24 04:19:33,975 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:19:33,990 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049?flat'} 2025-07-24 04:19:33,990 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049 virtual network link through opserver 10.0.0.38 2025-07-24 04:19:33,990 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:19:34,005 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049?flat'} 2025-07-24 04:19:34,005 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049 virtual network link through opserver 10.0.0.41 2025-07-24 04:19:34,005 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:19:34,022 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049?flat'} 2025-07-24 04:19:34,022 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-mgmt_vn-80255049 is found in opserver 2025-07-24 04:19:34,050 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:19:34,064 - DEBUG - VRF ids for VN ctest-mgmt_vn-80255049: {} 2025-07-24 04:19:34,361 - DEBUG - Response for create_network : {'network': {'id': '4d48d703-5269-4450-94c1-9007341a10fd', 'name': 'ctest-left_vn-02817852', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-left_vn-02817852'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:19:34,645 - DEBUG - Response for create_subnet : {'subnet': {'id': '66c43a43-8d2e-4080-b0d9-575d5872c90f', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': '4d48d703-5269-4450-94c1-9007341a10fd', 'ip_version': 4, 'cidr': '79.141.27.128/26', 'allocation_pools': [{'start': '79.141.27.130', 'end': '79.141.27.190'}], 'gateway_ip': '79.141.27.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '79.141.27.130', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:19:34,669 - INFO - Created VN ctest-left_vn-02817852 2025-07-24 04:19:34,687 - DEBUG - VN ctest-left_vn-02817852 UUID is 4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:34,789 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:34,799 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:34,819 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:34,932 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:34,949 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:34,961 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/83207c67-67d1-4ff3-854e-4f421150c807 2025-07-24 04:19:34,970 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/83207c67-67d1-4ff3-854e-4f421150c807 2025-07-24 04:19:34,979 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/86fcc0e7-d310-46bf-8247-5632ecca0314 2025-07-24 04:19:34,990 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:19:34,990 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:35,001 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/83207c67-67d1-4ff3-854e-4f421150c807 2025-07-24 04:19:35,023 - INFO - Verified VN network id 12 for VN 4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:35,024 - INFO - Verifications in API Server for VN ctest-left_vn-02817852 passed 2025-07-24 04:19:35,024 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:35,038 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/83207c67-67d1-4ff3-854e-4f421150c807 2025-07-24 04:19:35,050 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/83207c67-67d1-4ff3-854e-4f421150c807 2025-07-24 04:19:35,060 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/86fcc0e7-d310-46bf-8247-5632ecca0314 2025-07-24 04:19:35,077 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5568937348511122512', 'uuid-lslong': '10719006948731785469'}, 'enable': 'true', 'created': '2025-07-24T04:19:34', 'last-modified': '2025-07-24T04:19:34', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.498338'} 2025-07-24 04:19:35,087 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:19:35,095 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5568937348511122512', 'uuid-lslong': '10719006948731785469'}, 'enable': 'true', 'created': '2025-07-24T04:19:34', 'last-modified': '2025-07-24T04:19:34', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.509348'} 2025-07-24 04:19:35,103 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:19:35,110 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5568937348511122512', 'uuid-lslong': '10719006948731785469'}, 'enable': 'true', 'created': '2025-07-24T04:19:34', 'last-modified': '2025-07-24T04:19:34', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.530844'} 2025-07-24 04:19:35,120 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-24 04:19:35,120 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-02817852 passed 2025-07-24 04:19:35,120 - DEBUG - ====Verifying policy data for ctest-left_vn-02817852 in API_Server ====== 2025-07-24 04:19:35,120 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:35,128 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:35,144 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:35,255 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/4d48d703-5269-4450-94c1-9007341a10fd 2025-07-24 04:19:35,277 - DEBUG - =>VN ctest-left_vn-02817852 has no policy to be verified 2025-07-24 04:19:35,277 - DEBUG - Verifying the vn in opserver 2025-07-24 04:19:35,277 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852 virtual network link through opserver 10.0.0.19 2025-07-24 04:19:35,277 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:19:35,295 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852?flat'} 2025-07-24 04:19:35,295 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852 virtual network link through opserver 10.0.0.38 2025-07-24 04:19:35,295 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:19:35,310 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852?flat'} 2025-07-24 04:19:35,310 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852 virtual network link through opserver 10.0.0.41 2025-07-24 04:19:35,310 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:19:35,325 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852?flat'} 2025-07-24 04:19:35,325 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-left_vn-02817852 is found in opserver 2025-07-24 04:19:35,356 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:19:35,371 - DEBUG - VRF ids for VN ctest-left_vn-02817852: {} 2025-07-24 04:19:35,741 - DEBUG - Response for create_network : {'network': {'id': '6183c17f-32e6-4e79-85a8-1949a75ba2f4', 'name': 'ctest-right_vn-96803066', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-right_vn-96803066'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:19:35,995 - DEBUG - Response for create_subnet : {'subnet': {'id': '341cba02-2c42-4f38-b49e-ad9ed4178274', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': '6183c17f-32e6-4e79-85a8-1949a75ba2f4', 'ip_version': 4, 'cidr': '29.205.13.64/26', 'allocation_pools': [{'start': '29.205.13.66', 'end': '29.205.13.126'}], 'gateway_ip': '29.205.13.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '29.205.13.66', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:19:36,023 - INFO - Created VN ctest-right_vn-96803066 2025-07-24 04:19:36,039 - DEBUG - VN ctest-right_vn-96803066 UUID is 6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,139 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:36,147 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:36,167 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:36,283 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,303 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,320 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/fed217f3-f59c-4792-8d72-b20c0a3bfbfe 2025-07-24 04:19:36,332 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/fed217f3-f59c-4792-8d72-b20c0a3bfbfe 2025-07-24 04:19:36,345 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/6766bac0-af9e-4212-9ace-f81b3ddb010e 2025-07-24 04:19:36,356 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:19:36,356 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,372 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/fed217f3-f59c-4792-8d72-b20c0a3bfbfe 2025-07-24 04:19:36,394 - INFO - Verified VN network id 13 for VN 6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,394 - INFO - Verifications in API Server for VN ctest-right_vn-96803066 passed 2025-07-24 04:19:36,395 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,411 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/fed217f3-f59c-4792-8d72-b20c0a3bfbfe 2025-07-24 04:19:36,420 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/fed217f3-f59c-4792-8d72-b20c0a3bfbfe 2025-07-24 04:19:36,447 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/6766bac0-af9e-4212-9ace-f81b3ddb010e 2025-07-24 04:19:36,466 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7026672595687067257', 'uuid-lslong': '9630975607262913268'}, 'enable': 'true', 'created': '2025-07-24T04:19:35', 'last-modified': '2025-07-24T04:19:35', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.540737'} 2025-07-24 04:19:36,477 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:19:36,485 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7026672595687067257', 'uuid-lslong': '9630975607262913268'}, 'enable': 'true', 'created': '2025-07-24T04:19:35', 'last-modified': '2025-07-24T04:19:35', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.560063'} 2025-07-24 04:19:36,493 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:19:36,503 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7026672595687067257', 'uuid-lslong': '9630975607262913268'}, 'enable': 'true', 'created': '2025-07-24T04:19:35', 'last-modified': '2025-07-24T04:19:35', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.583103'} 2025-07-24 04:19:36,512 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-24 04:19:36,512 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-96803066 passed 2025-07-24 04:19:36,512 - DEBUG - ====Verifying policy data for ctest-right_vn-96803066 in API_Server ====== 2025-07-24 04:19:36,512 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:36,519 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:36,540 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:36,665 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/6183c17f-32e6-4e79-85a8-1949a75ba2f4 2025-07-24 04:19:36,683 - DEBUG - =>VN ctest-right_vn-96803066 has no policy to be verified 2025-07-24 04:19:36,683 - DEBUG - Verifying the vn in opserver 2025-07-24 04:19:36,683 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066 virtual network link through opserver 10.0.0.19 2025-07-24 04:19:36,683 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:19:36,699 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066?flat'} 2025-07-24 04:19:36,699 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066 virtual network link through opserver 10.0.0.38 2025-07-24 04:19:36,700 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:19:36,717 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066?flat'} 2025-07-24 04:19:36,717 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066 virtual network link through opserver 10.0.0.41 2025-07-24 04:19:36,717 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:19:36,734 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066?flat'} 2025-07-24 04:19:36,734 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-right_vn-96803066 is found in opserver 2025-07-24 04:19:36,772 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:19:36,787 - DEBUG - VRF ids for VN ctest-right_vn-96803066: {} 2025-07-24 04:19:37,095 - DEBUG - Response for create_network : {'network': {'id': '97460080-deb9-4548-aaf8-6dabecead393', 'name': 'ctest-trans_left_vn-02774343', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-trans_left_vn-02774343'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:19:37,370 - DEBUG - Response for create_subnet : {'subnet': {'id': '45964dff-5d26-4597-adb4-82b1748ca20e', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': '97460080-deb9-4548-aaf8-6dabecead393', 'ip_version': 4, 'cidr': '86.67.134.128/26', 'allocation_pools': [{'start': '86.67.134.130', 'end': '86.67.134.190'}], 'gateway_ip': '86.67.134.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '86.67.134.130', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:19:37,407 - INFO - Created VN ctest-trans_left_vn-02774343 2025-07-24 04:19:37,422 - DEBUG - VN ctest-trans_left_vn-02774343 UUID is 97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:37,524 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:37,532 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:37,546 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:37,659 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:37,680 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:37,694 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/182b885b-4948-4abc-ac6d-4803dbd92056 2025-07-24 04:19:37,711 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/182b885b-4948-4abc-ac6d-4803dbd92056 2025-07-24 04:19:37,722 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/2d0db1dd-0c7c-4478-ab97-388e86596764 2025-07-24 04:19:37,733 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-24 04:19:37,733 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:37,744 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/182b885b-4948-4abc-ac6d-4803dbd92056 2025-07-24 04:19:37,769 - INFO - Verified VN network id 14 for VN 97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:37,769 - INFO - Verifications in API Server for VN ctest-trans_left_vn-02774343 passed 2025-07-24 04:19:37,769 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:37,791 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/182b885b-4948-4abc-ac6d-4803dbd92056 2025-07-24 04:19:37,802 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/182b885b-4948-4abc-ac6d-4803dbd92056 2025-07-24 04:19:37,812 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/2d0db1dd-0c7c-4478-ab97-388e86596764 2025-07-24 04:19:37,837 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10900400501589361992', 'uuid-lslong': '12319717365853639571'}, 'enable': 'true', 'created': '2025-07-24T04:19:37', 'last-modified': '2025-07-24T04:19:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.526016'} 2025-07-24 04:19:37,846 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-24 04:19:37,853 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10900400501589361992', 'uuid-lslong': '12319717365853639571'}, 'enable': 'true', 'created': '2025-07-24T04:19:37', 'last-modified': '2025-07-24T04:19:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.549723'} 2025-07-24 04:19:37,861 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-24 04:19:37,869 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10900400501589361992', 'uuid-lslong': '12319717365853639571'}, 'enable': 'true', 'created': '2025-07-24T04:19:37', 'last-modified': '2025-07-24T04:19:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.563555'} 2025-07-24 04:19:37,879 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-24 04:19:37,880 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_left_vn-02774343 passed 2025-07-24 04:19:37,880 - DEBUG - ====Verifying policy data for ctest-trans_left_vn-02774343 in API_Server ====== 2025-07-24 04:19:37,880 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:37,889 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:37,905 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:38,018 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/97460080-deb9-4548-aaf8-6dabecead393 2025-07-24 04:19:38,029 - DEBUG - =>VN ctest-trans_left_vn-02774343 has no policy to be verified 2025-07-24 04:19:38,030 - DEBUG - Verifying the vn in opserver 2025-07-24 04:19:38,030 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343 virtual network link through opserver 10.0.0.19 2025-07-24 04:19:38,030 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:19:38,046 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343?flat'} 2025-07-24 04:19:38,046 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343 virtual network link through opserver 10.0.0.38 2025-07-24 04:19:38,046 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:19:38,063 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343?flat'} 2025-07-24 04:19:38,063 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343 virtual network link through opserver 10.0.0.41 2025-07-24 04:19:38,064 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:19:38,084 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343?flat'} 2025-07-24 04:19:38,084 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-trans_left_vn-02774343 is found in opserver 2025-07-24 04:19:38,115 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:19:38,132 - DEBUG - VRF ids for VN ctest-trans_left_vn-02774343: {} 2025-07-24 04:19:38,464 - DEBUG - Response for create_network : {'network': {'id': '23ed7936-5147-4250-b378-1d0f674b6e5a', 'name': 'ctest-trans_right_vn-04853814', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-09960119', 'ctest-trans_right_vn-04853814'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:19:38,725 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c6871e54-22bd-4f20-99f1-456aeea0e2dd', 'name': '', 'tenant_id': '03192ce2e38a4b84a6a1abff34ba7a82', 'network_id': '23ed7936-5147-4250-b378-1d0f674b6e5a', 'ip_version': 4, 'cidr': '204.168.179.0/26', 'allocation_pools': [{'start': '204.168.179.2', 'end': '204.168.179.62'}], 'gateway_ip': '204.168.179.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '204.168.179.2', 'tags': [], 'project_id': '03192ce2e38a4b84a6a1abff34ba7a82'}} 2025-07-24 04:19:38,753 - INFO - Created VN ctest-trans_right_vn-04853814 2025-07-24 04:19:38,776 - DEBUG - VN ctest-trans_right_vn-04853814 UUID is 23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:38,862 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:38,870 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:38,886 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:38,998 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:39,018 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:39,032 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/3446c292-75e2-49ea-9a12-869f8380ea2f 2025-07-24 04:19:39,041 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/3446c292-75e2-49ea-9a12-869f8380ea2f 2025-07-24 04:19:39,056 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/a9a1ae4a-d87e-46e9-9602-a0bd1bb28a5a 2025-07-24 04:19:39,067 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-24 04:19:39,067 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:39,077 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/3446c292-75e2-49ea-9a12-869f8380ea2f 2025-07-24 04:19:39,099 - INFO - Verified VN network id 15 for VN 23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:39,100 - INFO - Verifications in API Server for VN ctest-trans_right_vn-04853814 passed 2025-07-24 04:19:39,100 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:39,113 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/3446c292-75e2-49ea-9a12-869f8380ea2f 2025-07-24 04:19:39,123 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/3446c292-75e2-49ea-9a12-869f8380ea2f 2025-07-24 04:19:39,140 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/a9a1ae4a-d87e-46e9-9602-a0bd1bb28a5a 2025-07-24 04:19:39,158 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2588858635006722640', 'uuid-lslong': '12932118281989090906'}, 'enable': 'true', 'created': '2025-07-24T04:19:38', 'last-modified': '2025-07-24T04:19:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.505548'} 2025-07-24 04:19:39,166 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-24 04:19:39,173 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2588858635006722640', 'uuid-lslong': '12932118281989090906'}, 'enable': 'true', 'created': '2025-07-24T04:19:38', 'last-modified': '2025-07-24T04:19:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.521064'} 2025-07-24 04:19:39,181 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-24 04:19:39,188 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-09960119', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '2588858635006722640', 'uuid-lslong': '12932118281989090906'}, 'enable': 'true', 'created': '2025-07-24T04:19:38', 'last-modified': '2025-07-24T04:19:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.532754'} 2025-07-24 04:19:39,197 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-24 04:19:39,197 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_right_vn-04853814 passed 2025-07-24 04:19:39,198 - DEBUG - ====Verifying policy data for ctest-trans_right_vn-04853814 in API_Server ====== 2025-07-24 04:19:39,198 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:19:39,205 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:19:39,220 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:19:39,325 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/23ed7936-5147-4250-b378-1d0f674b6e5a 2025-07-24 04:19:39,340 - DEBUG - =>VN ctest-trans_right_vn-04853814 has no policy to be verified 2025-07-24 04:19:39,340 - DEBUG - Verifying the vn in opserver 2025-07-24 04:19:39,340 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814 virtual network link through opserver 10.0.0.19 2025-07-24 04:19:39,340 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:19:39,359 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814?flat'} 2025-07-24 04:19:39,359 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814 virtual network link through opserver 10.0.0.38 2025-07-24 04:19:39,359 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:19:39,376 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814?flat'} 2025-07-24 04:19:39,376 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814 virtual network link through opserver 10.0.0.41 2025-07-24 04:19:39,377 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:19:39,398 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814?flat'} 2025-07-24 04:19:39,398 - INFO - Validated that VN default-domain:ctest-TestSvcHC-09960119:ctest-trans_right_vn-04853814 is found in opserver 2025-07-24 04:19:39,431 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:19:39,446 - DEBUG - VRF ids for VN ctest-trans_right_vn-04853814: {} 2025-07-24 04:19:40,775 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3712-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3712-2) 2025-07-24 04:19:42,350 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3712-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3712-1) 2025-07-24 04:19:42,351 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-22216617'] 2025-07-24 04:19:42,682 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-22216617'] created succesfully 2025-07-24 04:21:23,515 - DEBUG - Installing image tiny_trans_fw 2025-07-24 04:21:23,515 - DEBUG - Image info {'name': 'tinycore-net-transparent-fw.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-07-24 04:21:23,516 - DEBUG - Image build path file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-24 04:21:23,516 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-24 04:21:23,516 - DEBUG - Unzip image 2025-07-24 04:21:23,516 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-24 04:21:23,536 - DEBUG - Try to create image in user context 2025-07-24 04:21:23,601 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-22216617'] 2025-07-24 04:21:23,712 - INFO - Service template: ['default-domain', 'ctest-service_template_1-22216617'] deleted successfully. 2025-07-24 04:21:23,712 - INFO - Skip interface_detach for VM ctest-right_vm-89894193 in state BUILD 2025-07-24 04:21:23,712 - INFO - Deleting VM ctest-right_vm-89894193 2025-07-24 04:21:23,712 - ERROR - VM ctest-right_vm-89894193 has failed to come up 2025-07-24 04:21:23,712 - ERROR - Nova failed to get host of the VM 2025-07-24 04:21:23,712 - ERROR - VM ctest-right_vm-89894193 has failed to come up 2025-07-24 04:21:23,712 - ERROR - Nova failed to get host of the VM 2025-07-24 04:21:23,797 - INFO - Skip interface_detach for VM ctest-left_vm-80266763 in state BUILD 2025-07-24 04:21:23,797 - INFO - Deleting VM ctest-left_vm-80266763 2025-07-24 04:21:23,798 - ERROR - VM ctest-left_vm-80266763 has failed to come up 2025-07-24 04:21:23,799 - ERROR - Nova failed to get host of the VM 2025-07-24 04:21:23,799 - ERROR - VM ctest-left_vm-80266763 has failed to come up 2025-07-24 04:21:23,799 - ERROR - Nova failed to get host of the VM 2025-07-24 04:21:23,881 - INFO - Deleting VN ctest-trans_right_vn-04853814 2025-07-24 04:21:24,062 - DEBUG - Response for deleting network () 2025-07-24 04:21:24,062 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/3446c292-75e2-49ea-9a12-869f8380ea2f 2025-07-24 04:21:24,072 - DEBUG - Response Code: 404 2025-07-24 04:21:24,073 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:21:24,088 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:21:24,135 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:21:24,203 - INFO - Validated that VN ctest-trans_right_vn-04853814 is not found in API Server 2025-07-24 04:21:24,217 - DEBUG - VN ctest-trans_right_vn-04853814 is not present in Agent 10.0.0.57 2025-07-24 04:21:24,235 - DEBUG - VN ctest-trans_right_vn-04853814 is not present in Agent 10.0.0.23 2025-07-24 04:21:24,235 - INFO - Validated that VN ctest-trans_right_vn-04853814 is not in any agent 2025-07-24 04:21:24,328 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_right_vn-04853814 info 2025-07-24 04:21:24,328 - INFO - Deleting VN ctest-trans_left_vn-02774343 2025-07-24 04:21:24,582 - DEBUG - Response for deleting network () 2025-07-24 04:21:24,583 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/182b885b-4948-4abc-ac6d-4803dbd92056 2025-07-24 04:21:24,593 - DEBUG - Response Code: 404 2025-07-24 04:21:24,593 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:21:24,607 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:21:24,637 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:21:24,771 - INFO - Validated that VN ctest-trans_left_vn-02774343 is not found in API Server 2025-07-24 04:21:24,787 - DEBUG - VN ctest-trans_left_vn-02774343 is not present in Agent 10.0.0.57 2025-07-24 04:21:24,812 - DEBUG - VN ctest-trans_left_vn-02774343 is not present in Agent 10.0.0.23 2025-07-24 04:21:24,812 - INFO - Validated that VN ctest-trans_left_vn-02774343 is not in any agent 2025-07-24 04:21:24,891 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_left_vn-02774343 info 2025-07-24 04:21:24,891 - INFO - Deleting VN ctest-right_vn-96803066 2025-07-24 04:21:25,148 - DEBUG - Response for deleting network () 2025-07-24 04:21:25,148 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/fed217f3-f59c-4792-8d72-b20c0a3bfbfe 2025-07-24 04:21:25,165 - DEBUG - Response Code: 404 2025-07-24 04:21:25,165 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:21:25,180 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:21:25,194 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:21:25,301 - INFO - Validated that VN ctest-right_vn-96803066 is not found in API Server 2025-07-24 04:21:25,317 - DEBUG - VN ctest-right_vn-96803066 is not present in Agent 10.0.0.57 2025-07-24 04:21:25,329 - DEBUG - VN ctest-right_vn-96803066 is not present in Agent 10.0.0.23 2025-07-24 04:21:25,329 - INFO - Validated that VN ctest-right_vn-96803066 is not in any agent 2025-07-24 04:21:25,373 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-96803066 info 2025-07-24 04:21:25,374 - INFO - Deleting VN ctest-left_vn-02817852 2025-07-24 04:21:25,558 - DEBUG - Response for deleting network () 2025-07-24 04:21:25,558 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/83207c67-67d1-4ff3-854e-4f421150c807 2025-07-24 04:21:25,565 - DEBUG - Response Code: 404 2025-07-24 04:21:25,565 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:21:25,577 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:21:25,609 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:21:25,710 - INFO - Validated that VN ctest-left_vn-02817852 is not found in API Server 2025-07-24 04:21:25,732 - DEBUG - VN ctest-left_vn-02817852 is not present in Agent 10.0.0.57 2025-07-24 04:21:25,747 - DEBUG - VN ctest-left_vn-02817852 is not present in Agent 10.0.0.23 2025-07-24 04:21:25,747 - INFO - Validated that VN ctest-left_vn-02817852 is not in any agent 2025-07-24 04:21:25,791 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-02817852 info 2025-07-24 04:21:25,791 - INFO - Deleting VN ctest-mgmt_vn-80255049 2025-07-24 04:21:25,977 - DEBUG - Response for deleting network () 2025-07-24 04:21:25,978 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/40645a59-d120-4235-b1f3-504150d94f34 2025-07-24 04:21:25,995 - DEBUG - Response Code: 404 2025-07-24 04:21:25,995 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:21:26,005 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:21:26,025 - DEBUG - Requesting: http://10.0.0.19:8082/project/03192ce2-e38a-4b84-a6a1-abff34ba7a82 2025-07-24 04:21:26,125 - INFO - Validated that VN ctest-mgmt_vn-80255049 is not found in API Server 2025-07-24 04:21:26,147 - DEBUG - VN ctest-mgmt_vn-80255049 is not present in Agent 10.0.0.57 2025-07-24 04:21:26,165 - DEBUG - VN ctest-mgmt_vn-80255049 is not present in Agent 10.0.0.23 2025-07-24 04:21:26,165 - INFO - Validated that VN ctest-mgmt_vn-80255049 is not in any agent 2025-07-24 04:21:26,213 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-80255049 info 2025-07-24 04:21:27,499 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 137, in test_svc_trans_segment_right_hc_fail_svm
    assert self.svc_hc_fail_svm_common()
  File "/contrail-test/scripts/svc_health_check/test_svc_health_check.py", line 180, in svc_hc_fail_svm_common
    create_svms=True, max_inst=1)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-net-transparent-fw.qcow2'
2025-07-24 04:21:27,516 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 04:21:27,517 - INFO -
2025-07-24 04:21:27,517 - INFO - END TEST : test_svc_trans_segment_right_hc_fail_svm : FAILED[0:01:56]
2025-07-24 04:21:27,517 - INFO - -------------------------------------------------------------------------------- 2025-07-24 04:21:28,378 - INFO - Deleted project: ctest-TestSvcHC-09960119, ID : 03192ce2-e38a-4b84-a6a1-abff34ba7a82