2025-07-22 17:34:35,278 - INFO - Domain Default found not creating 2025-07-22 17:34:35,465 - INFO - Project ctest-AnalyticsTestSanityWithResource-73069084 not found, creating it 2025-07-22 17:34:36,072 - INFO - Created Project:ctest-AnalyticsTestSanityWithResource-73069084, ID : 800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:34:38,187 - DEBUG - Response for create_network : {'network': {'id': 'c968cbcd-a3dc-48ac-a21f-86caa3f01efa', 'name': 'ctest-vn1-76443075', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn1-76443075'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:34:38,517 - DEBUG - Response for create_subnet : {'subnet': {'id': '50a6e787-05af-4183-a0ed-9bd86597803b', 'name': '', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'network_id': 'c968cbcd-a3dc-48ac-a21f-86caa3f01efa', 'ip_version': 4, 'cidr': '207.35.205.128/26', 'allocation_pools': [{'start': '207.35.205.130', 'end': '207.35.205.190'}], 'gateway_ip': '207.35.205.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '207.35.205.130', 'tags': [], 'project_id': '800f200af78c4ee6bd7febd6bb3d970c'}} 2025-07-22 17:34:38,544 - INFO - Created VN ctest-vn1-76443075 2025-07-22 17:34:38,560 - DEBUG - VN ctest-vn1-76443075 UUID is c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:38,662 - DEBUG - Services list from nova: [, , , ] 2025-07-22 17:34:38,662 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:34:39,915 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 17:34:41,438 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 17:34:41,438 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:34:41,758 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:34:41,793 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:34:41,815 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:34:41,951 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:41,996 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:42,035 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:34:42,055 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:34:42,065 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:34:42,075 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:34:42,075 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:42,098 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:34:42,122 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:42,122 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:34:42,122 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:42,146 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:34:42,155 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:34:42,166 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:34:42,183 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:34:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:03.722769'} 2025-07-22 17:34:42,196 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:34:42,213 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:34:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:03.753213'} 2025-07-22 17:34:42,223 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:34:42,229 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:34:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:03.776557'} 2025-07-22 17:34:42,237 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:34:42,237 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:34:42,238 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:34:42,238 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:34:42,246 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:34:42,258 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:34:42,382 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:34:42,401 - DEBUG - =>VN ctest-vn1-76443075 has no policy to be verified 2025-07-22 17:34:42,401 - DEBUG - Verifying the vn in opserver 2025-07-22 17:34:42,402 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:34:42,402 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:34:42,706 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:34:42,769 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:34:42,769 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:34:42,769 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:34:43,073 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:34:43,139 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:34:43,139 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:34:43,139 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:34:43,431 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:34:43,498 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:34:43,498 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:34:43,514 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:34:43,514 - INFO - Waiting for VM ctest-vn1_vm1-61388391 to be up.. 2025-07-22 17:34:43,601 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:34:43,601 - INFO - VM name : ctest-vn1_vm1-61388391 2025-07-22 17:34:43,679 - DEBUG - VM ctest-vn1_vm1-61388391 ID is 34cb0ee0-029f-4264-a333-b7a666587fad 2025-07-22 17:34:43,704 - DEBUG - VM ctest-vn1_vm1-61388391 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 17:34:43,784 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/34cb0ee0-029f-4264-a333-b7a666587fad 2025-07-22 17:34:44,079 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/34cb0ee0-029f-4264-a333-b7a666587fad 2025-07-22 17:34:44,113 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/d291ca2f-cb39-4044-9e73-ba62914bac5b 2025-07-22 17:34:47,357 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1005ms') 2025-07-22 17:34:47,357 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-61388391 failed! 2025-07-22 17:34:47,374 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is 207.35.205.129 and allocation pool is NOT set 2025-07-22 17:34:51,449 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-07-22 17:34:51,449 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-61388391 failed! 2025-07-22 17:34:51,465 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is 207.35.205.129 and allocation pool is NOT set 2025-07-22 17:34:55,543 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 17:34:55,543 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-61388391 failed! 2025-07-22 17:34:55,560 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is 207.35.205.129 and allocation pool is NOT set 2025-07-22 17:34:59,640 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-07-22 17:34:59,640 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-61388391 failed! 2025-07-22 17:34:59,658 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is 207.35.205.129 and allocation pool is NOT set 2025-07-22 17:35:03,736 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-07-22 17:35:03,736 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-61388391 failed! 2025-07-22 17:35:03,755 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is 207.35.205.129 and allocation pool is NOT set 2025-07-22 17:35:05,819 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.20 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.567 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.567/1.882/3.198/1.315 ms') 2025-07-22 17:35:05,819 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-61388391 passed 2025-07-22 17:35:05,935 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:05,935 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-61388391, IP 207.35.205.131, Port 22 2025-07-22 17:35:06,003 - DEBUG - Error on ssh to cirros@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:35:06,086 - DEBUG - VM ctest-vn1_vm1-61388391 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:35:11,087 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:11,087 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-61388391, IP 207.35.205.131, Port 22 2025-07-22 17:35:11,155 - DEBUG - Error on ssh to cirros@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:35:11,240 - DEBUG - VM ctest-vn1_vm1-61388391 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:35:16,241 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:16,241 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-61388391, IP 207.35.205.131, Port 22 2025-07-22 17:35:16,411 - DEBUG - VM ctest-vn1_vm1-61388391 is ready for SSH connections 2025-07-22 17:35:16,411 - INFO - Waiting for VM ctest-vn1_vm2-64726636 to be up.. 2025-07-22 17:35:16,492 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:35:16,492 - INFO - VM name : ctest-vn1_vm2-64726636 2025-07-22 17:35:16,567 - DEBUG - VM ctest-vn1_vm2-64726636 ID is c2ae023f-a2d4-4a18-9107-96961a39aee4 2025-07-22 17:35:16,567 - DEBUG - VM ctest-vn1_vm2-64726636 launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 17:35:16,646 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/c2ae023f-a2d4-4a18-9107-96961a39aee4 2025-07-22 17:35:16,657 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/cb6f44c7-d38f-4bb3-bba6-843a4ba1eee1 2025-07-22 17:35:17,873 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=2.59 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.482 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.482/1.535/2.588/1.053 ms') 2025-07-22 17:35:17,873 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-vn1_vm2-64726636 passed 2025-07-22 17:35:17,996 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:17,996 - DEBUG - Waiting to SSH to VM ctest-vn1_vm2-64726636, IP 207.35.205.132, Port 22 2025-07-22 17:35:18,163 - DEBUG - VM ctest-vn1_vm2-64726636 is ready for SSH connections 2025-07-22 17:35:18,490 - DEBUG - Response for create_network : {'network': {'id': '52b2687d-c79c-48bf-a30a-03f8278977cf', 'name': 'ctest-vn2-59754775', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn2-59754775'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:35:18,781 - DEBUG - Response for create_subnet : {'subnet': {'id': '9064865b-ffb1-4a7a-8b3f-d6883e01b5e5', 'name': '', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'network_id': '52b2687d-c79c-48bf-a30a-03f8278977cf', 'ip_version': 4, 'cidr': '206.35.68.192/26', 'allocation_pools': [{'start': '206.35.68.194', 'end': '206.35.68.254'}], 'gateway_ip': '206.35.68.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '206.35.68.194', 'tags': [], 'project_id': '800f200af78c4ee6bd7febd6bb3d970c'}} 2025-07-22 17:35:18,810 - INFO - Created VN ctest-vn2-59754775 2025-07-22 17:35:18,828 - DEBUG - VN ctest-vn2-59754775 UUID is 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:19,199 - DEBUG - Response for create_network : {'network': {'id': 'abc36e5c-8e5c-41cf-bddf-537f447f09bf', 'name': 'ctest-fip_vn-19357511', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-fip_vn-19357511'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:35:19,454 - DEBUG - Response for create_subnet : {'subnet': {'id': '280f700c-c30a-4530-99cc-6e3e9e5ba5c3', 'name': '', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'network_id': 'abc36e5c-8e5c-41cf-bddf-537f447f09bf', 'ip_version': 4, 'cidr': '19.122.16.0/26', 'allocation_pools': [{'start': '19.122.16.2', 'end': '19.122.16.62'}], 'gateway_ip': '19.122.16.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '19.122.16.2', 'tags': [], 'project_id': '800f200af78c4ee6bd7febd6bb3d970c'}} 2025-07-22 17:35:19,477 - INFO - Created VN ctest-fip_vn-19357511 2025-07-22 17:35:19,491 - DEBUG - VN ctest-fip_vn-19357511 UUID is abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:19,569 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:35:20,878 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1) 2025-07-22 17:35:22,485 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1) 2025-07-22 17:35:24,224 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-2) 2025-07-22 17:35:24,224 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:35:24,233 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:35:24,254 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:35:24,353 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:35:24,374 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:35:24,390 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:35:24,400 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:35:24,410 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:35:24,420 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:35:24,420 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:35:24,432 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:35:24,455 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:35:24,455 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:35:24,455 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:35:24,470 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:35:24,480 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:35:24,490 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:35:24,509 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:34:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:46.047682'} 2025-07-22 17:35:24,518 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:35:24,525 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:34:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:46.065644'} 2025-07-22 17:35:24,534 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:35:24,542 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:34:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:46.088721'} 2025-07-22 17:35:24,551 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:35:24,551 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:35:24,551 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:35:24,551 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:35:24,560 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:35:24,574 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:35:24,704 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:35:24,728 - DEBUG - =>VN ctest-vn1-76443075 has no policy to be verified 2025-07-22 17:35:24,728 - DEBUG - Verifying the vn in opserver 2025-07-22 17:35:24,728 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:35:24,728 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:35:24,816 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:35:24,816 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:35:24,816 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:35:24,937 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:35:24,937 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:35:24,937 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:35:25,048 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:35:25,048 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:35:25,060 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:35:25,061 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:35:25,081 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:35:25,102 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:35:25,249 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:25,288 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:25,321 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:35:25,347 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:35:25,358 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:35:25,370 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:35:25,371 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:25,388 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:35:25,408 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:25,409 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:35:25,409 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:25,470 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:35:25,497 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:35:25,516 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:35:25,537 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:35:18', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.824001'} 2025-07-22 17:35:25,550 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:35:25,558 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:35:18', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.862501'} 2025-07-22 17:35:25,571 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:35:25,579 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:35:18', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.871868'} 2025-07-22 17:35:25,592 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:35:25,592 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:35:25,593 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:35:25,593 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:35:25,606 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:35:25,631 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:35:25,850 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:35:25,885 - DEBUG - =>VN ctest-vn2-59754775 has no policy to be verified 2025-07-22 17:35:25,885 - DEBUG - Verifying the vn in opserver 2025-07-22 17:35:25,885 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:35:25,885 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:35:25,911 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:35:25,911 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:35:25,911 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:35:25,941 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:35:25,941 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:35:25,941 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:35:26,090 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:35:26,090 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:35:26,104 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2'} 2025-07-22 17:35:26,104 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:35:26,120 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:35:26,166 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:35:26,355 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:26,391 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:26,421 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:35:26,459 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:35:26,481 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:35:26,496 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:35:26,496 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:26,514 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:35:26,548 - INFO - Verified VN network id 14 for VN abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:26,548 - INFO - Verifications in API Server for VN ctest-fip_vn-19357511 passed 2025-07-22 17:35:26,548 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:26,571 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:35:26,596 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:35:26,613 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:35:26,634 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:07.238978'} 2025-07-22 17:35:26,652 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:35:26,667 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:07.269537'} 2025-07-22 17:35:26,690 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:35:26,697 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:07.308440'} 2025-07-22 17:35:26,708 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:35:26,708 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-19357511 passed 2025-07-22 17:35:26,708 - DEBUG - ====Verifying policy data for ctest-fip_vn-19357511 in API_Server ====== 2025-07-22 17:35:26,708 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:35:26,721 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:35:26,743 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:35:26,878 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:35:26,897 - DEBUG - =>VN ctest-fip_vn-19357511 has no policy to be verified 2025-07-22 17:35:26,897 - DEBUG - Verifying the vn in opserver 2025-07-22 17:35:26,897 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.254 2025-07-22 17:35:26,897 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:35:27,011 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:35:27,011 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.50 2025-07-22 17:35:27,011 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:35:27,080 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:35:27,080 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.23 2025-07-22 17:35:27,080 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:35:27,098 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:35:27,098 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is found in opserver 2025-07-22 17:35:27,114 - DEBUG - VRF ids for VN ctest-fip_vn-19357511: {'10.0.0.19': '3'} 2025-07-22 17:35:27,114 - INFO - Waiting for VM ctest-fvn_vm1-13112516 to be up.. 2025-07-22 17:35:27,259 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:35:27,259 - INFO - VM name : ctest-fvn_vm1-13112516 2025-07-22 17:35:27,391 - DEBUG - VM ctest-fvn_vm1-13112516 ID is 225ef6c5-2112-41ef-b3a2-27d3864d496f 2025-07-22 17:35:27,391 - DEBUG - VM ctest-fvn_vm1-13112516 launched on Node an-jenkins-deploy-platform-ansible-os-3693-1 2025-07-22 17:35:27,501 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/225ef6c5-2112-41ef-b3a2-27d3864d496f 2025-07-22 17:35:27,515 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/f13daea0-96c2-40f6-800a-081111f2bede 2025-07-22 17:35:30,765 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms') 2025-07-22 17:35:30,765 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-13112516 failed! 2025-07-22 17:35:30,781 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is 19.122.16.1 and allocation pool is NOT set 2025-07-22 17:35:34,852 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1004ms') 2025-07-22 17:35:34,852 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-13112516 failed! 2025-07-22 17:35:34,870 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is 19.122.16.1 and allocation pool is NOT set 2025-07-22 17:35:38,945 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-07-22 17:35:38,946 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-13112516 failed! 2025-07-22 17:35:38,963 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is 19.122.16.1 and allocation pool is NOT set 2025-07-22 17:35:43,048 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-07-22 17:35:43,048 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-13112516 failed! 2025-07-22 17:35:43,065 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is 19.122.16.1 and allocation pool is NOT set 2025-07-22 17:35:47,137 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-07-22 17:35:47,138 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-13112516 failed! 2025-07-22 17:35:47,155 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is 19.122.16.1 and allocation pool is NOT set 2025-07-22 17:35:49,217 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=5.00 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.443 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.443/2.723/5.003/2.280 ms') 2025-07-22 17:35:49,217 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-13112516 passed 2025-07-22 17:35:49,330 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:49,330 - DEBUG - Waiting to SSH to VM ctest-fvn_vm1-13112516, IP 19.122.16.3, Port 22 2025-07-22 17:35:49,389 - DEBUG - Error on ssh to cirros@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:35:49,472 - DEBUG - VM ctest-fvn_vm1-13112516 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:35:54,473 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:54,473 - DEBUG - Waiting to SSH to VM ctest-fvn_vm1-13112516, IP 19.122.16.3, Port 22 2025-07-22 17:35:54,539 - DEBUG - Error on ssh to cirros@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-07-22 17:35:54,617 - DEBUG - VM ctest-fvn_vm1-13112516 is NOT ready for SSH connections, VM status: ACTIVE 2025-07-22 17:35:59,618 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:35:59,618 - DEBUG - Waiting to SSH to VM ctest-fvn_vm1-13112516, IP 19.122.16.3, Port 22 2025-07-22 17:35:59,791 - DEBUG - VM ctest-fvn_vm1-13112516 is ready for SSH connections 2025-07-22 17:35:59,791 - INFO - Waiting for VM ctest-vn2_vm2-24845532 to be up.. 2025-07-22 17:35:59,881 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:35:59,881 - INFO - VM name : ctest-vn2_vm2-24845532 2025-07-22 17:35:59,976 - DEBUG - VM ctest-vn2_vm2-24845532 ID is cd2258a2-cef4-4973-82de-a6f584bf4903 2025-07-22 17:35:59,976 - DEBUG - VM ctest-vn2_vm2-24845532 launched on Node an-jenkins-deploy-platform-ansible-os-3693-1 2025-07-22 17:36:00,066 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/cd2258a2-cef4-4973-82de-a6f584bf4903 2025-07-22 17:36:00,079 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/c6abcc5a-92b7-4a7d-b47e-f700f04dc52d 2025-07-22 17:36:01,293 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=2.87 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.472 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.472/1.670/2.868/1.198 ms') 2025-07-22 17:36:01,293 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn2_vm2-24845532 passed 2025-07-22 17:36:01,409 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:01,409 - DEBUG - Waiting to SSH to VM ctest-vn2_vm2-24845532, IP 206.35.68.195, Port 22 2025-07-22 17:36:01,587 - DEBUG - VM ctest-vn2_vm2-24845532 is ready for SSH connections 2025-07-22 17:36:01,587 - INFO - Waiting for VM mltf_vm to be up.. 2025-07-22 17:36:01,672 - DEBUG - VM is in ACTIVE state now 2025-07-22 17:36:01,672 - INFO - VM name : mltf_vm 2025-07-22 17:36:01,755 - DEBUG - VM mltf_vm ID is a7a6caea-6408-4542-b3d8-5206a5b65f14 2025-07-22 17:36:01,755 - DEBUG - VM mltf_vm launched on Node an-jenkins-deploy-platform-ansible-os-3693-2 2025-07-22 17:36:01,916 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a7a6caea-6408-4542-b3d8-5206a5b65f14 2025-07-22 17:36:01,929 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/79f148d1-d0b0-4d06-81fe-f0c073a28895 2025-07-22 17:36:01,942 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/ab1c26e8-0d76-4117-90a1-33ab58fc9a6a 2025-07-22 17:36:03,183 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n64 bytes from 169.254.0.5: icmp_seq=1 ttl=63 time=1.68 ms\r\n64 bytes from 169.254.0.5: icmp_seq=2 ttl=63 time=0.462 ms\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.462/1.069/1.676/0.607 ms') 2025-07-22 17:36:03,183 - INFO - Ping to Metadata IP 169.254.0.5 of VM mltf_vm passed 2025-07-22 17:36:03,302 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:03,302 - DEBUG - Waiting to SSH to VM mltf_vm, IP 207.35.205.133, Port 22 2025-07-22 17:36:03,466 - DEBUG - VM mltf_vm is ready for SSH connections 2025-07-22 17:36:03,582 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:03,582 - DEBUG - Waiting to SSH to VM mltf_vm, IP 207.35.205.133, Port 22 2025-07-22 17:36:03,741 - DEBUG - VM mltf_vm is ready for SSH connections 2025-07-22 17:36:03,742 - INFO - ================================================================================ 2025-07-22 17:36:03,743 - INFO - STARTING TEST : test_verify_session_record_table_inter_vn 2025-07-22 17:36:03,743 - INFO - TEST DESCRIPTION : Verify session record table ,generated stats between different vns 1.query and verify number of client session records 2.query and verify number of server session records 3.query with local_ip server_port protocol 4.query with server_port local_ip filter by server_port 5.query with client_port remote_ip filter by client_port Total we get three record limit by 2 6.query with sort_fields 2025-07-22 17:36:05,005 - DEBUG - Skipping xmpp flap check 2025-07-22 17:36:05,005 - INFO - Initial checks done. Running the testcase now 2025-07-22 17:36:05,005 - INFO - 2025-07-22 17:36:05,220 - INFO - Setting flow export rate: 100 2025-07-22 17:36:05,268 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = udp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-22 17:36:05,626 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'c968cbcd-a3dc-48ac-a21f-86caa3f01efa', 'name': 'ctest-vn1-76443075', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['50a6e787-05af-4183-a0ed-9bd86597803b'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn1-76443075'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '207.35.205.128/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:36:05,830 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '52b2687d-c79c-48bf-a30a-03f8278977cf', 'name': 'ctest-vn2-59754775', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['9064865b-ffb1-4a7a-8b3f-d6883e01b5e5'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn2-59754775'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '206.35.68.192/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:36:05,931 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:05,939 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:05,954 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:06,121 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:06,152 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:06,176 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:06,194 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:06,205 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:06,213 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:06,214 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:06,225 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:06,246 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:06,247 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:36:06,247 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:06,260 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:06,273 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:06,301 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:06,365 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.769731'} 2025-07-22 17:36:06,401 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:06,409 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.827423'} 2025-07-22 17:36:06,420 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:06,431 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.848122'} 2025-07-22 17:36:06,441 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:06,441 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:36:06,441 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:36:06,441 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:06,453 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:06,467 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:06,586 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:06,612 - DEBUG - ==>Verifying data for policy with id: 2ee68600-c722-4d08-b582-fe97431599cf, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:06,612 - INFO - VN ctest-vn1-76443075 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:06,612 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:06,612 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:06,612 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:06,632 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:06,632 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:06,632 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:06,650 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:06,650 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:06,650 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:06,670 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:06,670 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:36:06,683 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:36:06,683 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:06,691 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:06,705 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:06,815 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:06,835 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:06,850 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:06,866 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:06,879 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:06,892 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:06,892 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:06,906 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:06,926 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:06,926 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:36:06,926 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:06,939 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:06,948 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:06,957 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:06,974 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.174934'} 2025-07-22 17:36:06,984 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:06,993 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.195890'} 2025-07-22 17:36:07,004 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:07,014 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.213820'} 2025-07-22 17:36:07,026 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:07,026 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:36:07,026 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:36:07,026 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:07,036 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:07,052 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:07,151 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:07,164 - DEBUG - ==>Verifying data for policy with id: 2ee68600-c722-4d08-b582-fe97431599cf, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:07,164 - INFO - VN ctest-vn2-59754775 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:07,164 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:07,165 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:07,165 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:07,180 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:07,180 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:07,180 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:07,196 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:07,196 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:07,196 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:07,212 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:07,212 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:36:07,225 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2', '10.0.0.27': '3'} 2025-07-22 17:36:07,225 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:07,232 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:07,244 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:07,348 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:07,366 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:07,383 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:07,395 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:07,404 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:07,412 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:07,412 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:07,428 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:07,448 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:07,448 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:36:07,448 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:07,461 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:07,471 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:07,481 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:07,498 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.903259'} 2025-07-22 17:36:07,507 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:07,515 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.934845'} 2025-07-22 17:36:07,524 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:07,530 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.948514'} 2025-07-22 17:36:07,538 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:07,538 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:36:07,538 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:36:07,539 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:07,545 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:07,559 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:07,653 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:07,670 - DEBUG - ==>Verifying data for policy with id: 2ee68600-c722-4d08-b582-fe97431599cf, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:07,670 - INFO - VN ctest-vn1-76443075 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:07,670 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:07,670 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:07,670 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:07,684 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:07,685 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:07,685 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:07,700 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:07,700 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:07,700 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:07,718 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:07,718 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:36:07,731 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:36:07,731 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:07,740 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:07,754 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:07,854 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:07,871 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:07,884 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:07,894 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:07,905 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:07,915 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:07,915 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:07,928 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:07,960 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:07,960 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:36:07,961 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:07,976 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:07,988 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:08,000 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:08,020 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.222115'} 2025-07-22 17:36:08,032 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:08,040 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.242215'} 2025-07-22 17:36:08,048 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:08,057 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.257939'} 2025-07-22 17:36:08,066 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:08,066 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:36:08,066 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:36:08,066 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:08,073 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:08,087 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:08,188 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:08,201 - DEBUG - ==>Verifying data for policy with id: 2ee68600-c722-4d08-b582-fe97431599cf, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:08,201 - INFO - VN ctest-vn2-59754775 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:08,201 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:08,201 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:08,201 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:08,218 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:08,218 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:08,218 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:08,238 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:08,238 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:08,238 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:08,257 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:08,258 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:36:08,274 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2', '10.0.0.27': '3'} 2025-07-22 17:36:08,274 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:08,282 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:08,297 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:08,406 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:08,418 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:08,430 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:08,439 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:08,448 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:36:08,457 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:08,457 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:08,468 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:08,489 - INFO - Verified VN network id 14 for VN abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:08,490 - INFO - Verifications in API Server for VN ctest-fip_vn-19357511 passed 2025-07-22 17:36:08,490 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:08,503 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:08,512 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:08,523 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:36:08,538 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:49.142915'} 2025-07-22 17:36:08,546 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:08,554 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:49.159933'} 2025-07-22 17:36:08,563 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:08,570 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:49.181283'} 2025-07-22 17:36:08,577 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:08,577 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-19357511 passed 2025-07-22 17:36:08,577 - DEBUG - ====Verifying policy data for ctest-fip_vn-19357511 in API_Server ====== 2025-07-22 17:36:08,577 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:08,585 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:08,598 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:08,711 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:08,724 - DEBUG - =>VN ctest-fip_vn-19357511 has no policy to be verified 2025-07-22 17:36:08,724 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:08,724 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:08,724 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:08,738 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:36:08,738 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:08,738 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:08,755 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:36:08,755 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:08,755 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:08,771 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:36:08,771 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is found in opserver 2025-07-22 17:36:08,783 - DEBUG - VRF ids for VN ctest-fip_vn-19357511: {'10.0.0.19': '3'} 2025-07-22 17:36:08,784 - INFO - start time= 1753205768784074 2025-07-22 17:36:08,784 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:12,190 - DEBUG - file copy failed retrying 2025-07-22 17:36:15,582 - DEBUG - file copy failed retrying 2025-07-22 17:36:18,967 - DEBUG - file copy failed retrying 2025-07-22 17:36:22,346 - DEBUG - file copy failed retrying 2025-07-22 17:36:22,346 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:22,346 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoserver.py to 206.35.68.195 2025-07-22 17:36:22,347 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-99171913.pid --stats_file /tmp/server-ctest-random-99171913.stats 0<&- &> /tmp/server-ctest-random-99171913.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:36:22,347 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-99171913.pid --stats_file /tmp/server-ctest-random-99171913.stats 0<&- &> /tmp/server-ctest-random-99171913.log & 2025-07-22 17:36:22,935 - DEBUG - None 2025-07-22 17:36:26,662 - DEBUG - file copy failed retrying 2025-07-22 17:36:30,106 - DEBUG - file copy failed retrying 2025-07-22 17:36:33,498 - DEBUG - file copy failed retrying 2025-07-22 17:36:36,918 - DEBUG - file copy failed retrying 2025-07-22 17:36:36,918 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:36,919 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoclient.py to 207.35.205.131 2025-07-22 17:36:36,919 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-99171913.pid --stats_file /tmp/client-ctest-random-99171913.stats 0<&- &> /tmp/client-ctest-random-99171913.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:36:36,919 - DEBUG - nohup python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-99171913.pid --stats_file /tmp/client-ctest-random-99171913.stats 0<&- &> /tmp/client-ctest-random-99171913.log & 2025-07-22 17:36:37,464 - DEBUG - None 2025-07-22 17:36:37,464 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:37,464 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:37,464 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 206.35.68.195 2025-07-22 17:36:37,464 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-15993095.pid --stats_file /tmp/server-ctest-random-15993095.stats 0<&- &> /tmp/server-ctest-random-15993095.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:36:37,464 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-15993095.pid --stats_file /tmp/server-ctest-random-15993095.stats 0<&- &> /tmp/server-ctest-random-15993095.log & 2025-07-22 17:36:37,970 - DEBUG - None 2025-07-22 17:36:37,971 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:37,971 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 207.35.205.131 2025-07-22 17:36:37,971 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-15993095.pid --stats_file /tmp/client-ctest-random-15993095.stats 0<&- &> /tmp/client-ctest-random-15993095.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:36:37,971 - DEBUG - nohup python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-15993095.pid --stats_file /tmp/client-ctest-random-15993095.stats 0<&- &> /tmp/client-ctest-random-15993095.log & 2025-07-22 17:36:38,217 - DEBUG - None 2025-07-22 17:36:38,218 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:38,218 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:38,218 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 206.35.68.195 2025-07-22 17:36:38,218 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-53436576.pid --stats_file /tmp/server-ctest-random-53436576.stats 0<&- &> /tmp/server-ctest-random-53436576.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:36:38,218 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-53436576.pid --stats_file /tmp/server-ctest-random-53436576.stats 0<&- &> /tmp/server-ctest-random-53436576.log & 2025-07-22 17:36:38,456 - DEBUG - None 2025-07-22 17:36:38,457 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:36:38,457 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 207.35.205.131 2025-07-22 17:36:38,457 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-53436576.pid --stats_file /tmp/client-ctest-random-53436576.stats 0<&- &> /tmp/client-ctest-random-53436576.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:36:38,457 - DEBUG - nohup python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-53436576.pid --stats_file /tmp/client-ctest-random-53436576.stats 0<&- &> /tmp/client-ctest-random-53436576.log & 2025-07-22 17:36:38,691 - DEBUG - None 2025-07-22 17:36:48,692 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-99171913.pid); sync; cat /tmp/client-ctest-random-99171913.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:36:48,692 - DEBUG - kill $(cat /tmp/client-ctest-random-99171913.pid); sync; cat /tmp/client-ctest-random-99171913.stats 2025-07-22 17:36:49,015 - DEBUG - cat: can't open '/tmp/client-ctest-random-99171913.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/client-ctest-random-99171913.stats': No such file or directory 2025-07-22 17:36:49,015 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-99171913.pid); sync; cat /tmp/server-ctest-random-99171913.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:36:49,015 - DEBUG - kill $(cat /tmp/server-ctest-random-99171913.pid); sync; cat /tmp/server-ctest-random-99171913.stats 2025-07-22 17:36:49,358 - DEBUG - cat: can't open '/tmp/server-ctest-random-99171913.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/server-ctest-random-99171913.stats': No such file or directory 2025-07-22 17:36:49,358 - INFO - Client - Sent: 0, Received: 0 2025-07-22 17:36:49,358 - INFO - Server - Sent: 0, Received: 0 2025-07-22 17:36:49,849 - INFO - Deleted policy policy1 2025-07-22 17:36:50,012 - INFO - Setting flow export rate: None 2025-07-22 17:36:52,705 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Tue Jul 22 17:36:49 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/analytics/test_analytics_resource.py in test_verify_session_record_table_inter_vn(self=) 1373 start_time = self.analytics_obj.getstarttime(vm_node_ip) 1374 self.logger.info("start time= %s" % (start_time)) 1375 self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) 1376 time.sleep(100) 1377 src_vn = self.res.vn1_fixture.vn_fq_name self = self.setup_and_create_streams = > self.res = self.res.vn1_vm1_fixture = self.res.vn2_vm2_fixture = /contrail-test/serial_scripts/analytics/base.py in setup_and_create_streams(self=, src_vm=, dst_vm=, sport=8000, dport=9003, count=100) 168 time.sleep(10) 169 for traffic_obj in traffic_objs: 170 self.stop_traffic(traffic_obj) 171 172 #end setup_create_streams self = self.stop_traffic = > traffic_obj = /contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False) 796 assert False, "No drops observed in packets %s"%msg 797 elif expectation: 798 assert sent and recv, "Packets not %s"%msg 799 if recv*100/float(sent) < 90: 800 assert False, "Packets not %s"%msg sent = 0 recv = 0 msg = 'transferred between 207.35.205.131 and 206.35.68.195, proto udp sport 8000 and dport 9000' AssertionError: Packets not transferred between 207.35.205.131 and 206.35.68.195, proto udp sport 8000 and dport 9000 __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('Packets not transferred between 207.35.205.131 a...06.35.68.195, proto udp sport 8000 and dport 9000',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/analytics/test_analytics_resource.py", line 1375, in test_verify_session_record_table_inter_vn self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) File "/contrail-test/serial_scripts/analytics/base.py", line 170, in setup_and_create_streams self.stop_traffic(traffic_obj) File "/contrail-test/common/base.py", line 798, in stop_traffic assert sent and recv, "Packets not %s"%msg AssertionError: Packets not transferred between 207.35.205.131 and 206.35.68.195, proto udp sport 8000 and dport 9000 2025-07-22 17:36:52,705 - DEBUG - Skipping xmpp flap check 2025-07-22 17:36:52,705 - INFO - 2025-07-22 17:36:52,705 - INFO - END TEST : test_verify_session_record_table_inter_vn : FAILED[0:00:49] 2025-07-22 17:36:52,705 - INFO - -------------------------------------------------------------------------------- 2025-07-22 17:36:52,709 - INFO - ================================================================================ 2025-07-22 17:36:52,709 - INFO - STARTING TEST : test_verify_session_sampling_teardown 2025-07-22 17:36:52,709 - INFO - TEST DESCRIPTION : 1.query client session samples 2.query server session samples 3.query client session to get number of sessions exported 4.query session record table for teardown bytes/pkts 5.query sample count after teardown on server side 2025-07-22 17:36:53,961 - DEBUG - Skipping xmpp flap check 2025-07-22 17:36:53,961 - INFO - Initial checks done. Running the testcase now 2025-07-22 17:36:53,962 - INFO - 2025-07-22 17:36:54,184 - INFO - Setting flow export rate: 100 2025-07-22 17:36:54,232 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-22 17:36:54,596 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'c968cbcd-a3dc-48ac-a21f-86caa3f01efa', 'name': 'ctest-vn1-76443075', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['50a6e787-05af-4183-a0ed-9bd86597803b'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn1-76443075'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '207.35.205.128/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:36:54,803 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '52b2687d-c79c-48bf-a30a-03f8278977cf', 'name': 'ctest-vn2-59754775', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['9064865b-ffb1-4a7a-8b3f-d6883e01b5e5'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn2-59754775'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '206.35.68.192/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:36:54,935 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:54,957 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:54,984 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:55,100 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:55,119 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:55,133 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:55,144 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:55,155 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:55,164 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:55,164 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:55,175 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:55,200 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:55,200 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:36:55,200 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:55,214 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:55,225 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:55,236 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:55,250 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.694048'} 2025-07-22 17:36:55,257 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:55,264 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.717708'} 2025-07-22 17:36:55,272 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:55,279 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.731970'} 2025-07-22 17:36:55,287 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:55,287 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:36:55,287 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:36:55,288 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:55,295 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:55,313 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:55,418 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:55,434 - DEBUG - ==>Verifying data for policy with id: 366a3a3a-f22f-4659-a813-8460503e2cbb, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:55,434 - INFO - VN ctest-vn1-76443075 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:55,434 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:55,434 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:55,434 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:55,449 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:55,449 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:55,449 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:55,466 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:55,466 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:55,466 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:55,483 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:55,483 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:36:55,498 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:36:55,498 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:55,505 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:55,518 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:55,621 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:55,637 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:55,651 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:55,663 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:55,671 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:55,680 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:55,680 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:55,693 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:55,714 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:55,714 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:36:55,714 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:55,728 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:55,738 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:55,747 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:55,761 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.998283'} 2025-07-22 17:36:55,769 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:55,776 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.010270'} 2025-07-22 17:36:55,787 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:55,794 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.030081'} 2025-07-22 17:36:55,802 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:55,802 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:36:55,803 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:36:55,803 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:55,810 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:55,824 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:55,918 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:55,933 - DEBUG - ==>Verifying data for policy with id: 366a3a3a-f22f-4659-a813-8460503e2cbb, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:55,933 - INFO - VN ctest-vn2-59754775 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:55,935 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:55,935 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:55,935 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:55,951 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:55,951 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:55,951 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:55,967 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:55,967 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:55,967 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:55,982 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:55,982 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:36:55,998 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2', '10.0.0.27': '3'} 2025-07-22 17:36:55,998 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:56,007 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:56,022 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:56,139 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:56,153 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:56,168 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:56,180 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:56,192 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:56,203 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:56,203 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:56,218 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:56,240 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:56,240 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:36:56,240 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:56,253 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:56,263 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:36:56,273 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:36:56,290 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.732647'} 2025-07-22 17:36:56,298 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:56,306 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.758663'} 2025-07-22 17:36:56,314 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:56,322 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.775543'} 2025-07-22 17:36:56,331 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:36:56,331 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:36:56,331 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:36:56,331 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:56,340 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:56,352 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:56,460 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:36:56,472 - DEBUG - ==>Verifying data for policy with id: 366a3a3a-f22f-4659-a813-8460503e2cbb, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:56,472 - INFO - VN ctest-vn1-76443075 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:56,472 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:56,472 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:56,472 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:56,486 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:56,486 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:56,486 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:56,505 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:56,505 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:56,505 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:56,520 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:36:56,520 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:36:56,532 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:36:56,532 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:56,539 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:56,552 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:56,650 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:56,664 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:56,676 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:56,686 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:56,695 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:56,704 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:56,704 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:56,716 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:56,738 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:56,738 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:36:56,738 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:56,751 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:56,761 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:36:56,770 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:36:56,785 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.021606'} 2025-07-22 17:36:56,793 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:56,799 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.033571'} 2025-07-22 17:36:56,807 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:56,814 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:36:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.049663'} 2025-07-22 17:36:56,822 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:36:56,822 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:36:56,822 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:36:56,822 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:56,829 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:56,843 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:56,945 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:36:56,959 - DEBUG - ==>Verifying data for policy with id: 366a3a3a-f22f-4659-a813-8460503e2cbb, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:36:56,959 - INFO - VN ctest-vn2-59754775 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:36:56,959 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:56,959 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:56,959 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:56,977 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:56,977 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:56,977 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:56,992 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:56,992 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:56,993 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:57,009 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:36:57,009 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:36:57,024 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2', '10.0.0.27': '3'} 2025-07-22 17:36:57,024 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:57,031 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:57,045 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:57,166 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:57,182 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:57,200 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:57,218 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:57,228 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:36:57,240 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:57,240 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:57,252 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:57,279 - INFO - Verified VN network id 14 for VN abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:57,279 - INFO - Verifications in API Server for VN ctest-fip_vn-19357511 passed 2025-07-22 17:36:57,280 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:57,297 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:57,307 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:36:57,317 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:36:57,334 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:37.938144'} 2025-07-22 17:36:57,342 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:57,349 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:37.954990'} 2025-07-22 17:36:57,357 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:57,365 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:37.975846'} 2025-07-22 17:36:57,373 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:36:57,373 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-19357511 passed 2025-07-22 17:36:57,373 - DEBUG - ====Verifying policy data for ctest-fip_vn-19357511 in API_Server ====== 2025-07-22 17:36:57,373 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:36:57,382 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:36:57,397 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:36:57,529 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:36:57,541 - DEBUG - =>VN ctest-fip_vn-19357511 has no policy to be verified 2025-07-22 17:36:57,541 - DEBUG - Verifying the vn in opserver 2025-07-22 17:36:57,542 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.254 2025-07-22 17:36:57,542 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:36:57,560 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:36:57,560 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.50 2025-07-22 17:36:57,560 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:36:57,575 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:36:57,575 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.23 2025-07-22 17:36:57,575 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:36:57,591 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:36:57,591 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is found in opserver 2025-07-22 17:36:57,605 - DEBUG - VRF ids for VN ctest-fip_vn-19357511: {'10.0.0.19': '3'} 2025-07-22 17:36:57,606 - INFO - start time= 1753205817606057 2025-07-22 17:36:57,606 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 206.35.68.195, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:36:57,606 - DEBUG - ping -s 56 -c 3 -W 1 206.35.68.195 2025-07-22 17:36:59,911 - DEBUG - PING 206.35.68.195 (206.35.68.195): 56 data bytes 64 bytes from 206.35.68.195: seq=0 ttl=63 time=4.703 ms 64 bytes from 206.35.68.195: seq=1 ttl=63 time=1.240 ms 64 bytes from 206.35.68.195: seq=2 ttl=63 time=1.264 ms --- 206.35.68.195 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 1.240/2.402/4.703 ms 2025-07-22 17:36:59,912 - INFO - Ping to IP 206.35.68.195 from VM ctest-vn1_vm1-61388391 passed 2025-07-22 17:37:09,913 - INFO - Verify session samples and teardown pkts 2025-07-22 17:37:09,913 - INFO - Verifying SessionSeriesTable through opserver 10.0.0.254 2025-07-22 17:37:09,917 - DEBUG - Posting: http://10.0.0.254:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1753205817606057, 'end_time': 1753205829917141, 'select_fields': ['T'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-07-22 17:37:10,936 - DEBUG - [{'T': 1753205818404491}, {'T': 1753205819404712}, {'T': 1753205820404794}] 2025-07-22 17:37:10,936 - DEBUG - Posting: http://10.0.0.254:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1753205817606057, 'end_time': 1753205830936524, 'select_fields': ['T'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'server'} 2025-07-22 17:37:11,959 - DEBUG - [{'T': 1753205818178070}, {'T': 1753205819178135}, {'T': 1753205820178226}] 2025-07-22 17:37:11,960 - DEBUG - Posting: http://10.0.0.254:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1753205817606057, 'end_time': 1753205831960120, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-07-22 17:37:12,981 - DEBUG - [{'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'sample_count': 3}] 2025-07-22 17:37:12,981 - INFO - wait for the flows to get expire 2025-07-22 17:40:32,982 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-3693-2:Compute:contrail-vrouter-agent:0?flat 2025-07-22 17:40:33,085 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3693-2?flat 2025-07-22 17:40:33,167 - DEBUG - Posting: http://10.0.0.254:8081/analytics/query, payload {'table': 'SessionRecordTable', 'start_time': 1753205817606057, 'end_time': 1753206033167480, 'select_fields': ['vn', 'remote_vn', 'forward_teardown_pkts', 'reverse_teardown_pkts'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-07-22 17:40:34,251 - DEBUG - [{'forward_flow_uuid': '0f738bf7-f0e1-47af-8b3b-0de1f2950d62', 'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'reverse_flow_uuid': '5abc195a-dc9d-48dd-87b2-ae7b5fd192ba', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'forward_teardown_pkts': 3, 'reverse_teardown_pkts': 3}] 2025-07-22 17:40:34,251 - DEBUG - Posting: http://10.0.0.254:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1753205817606057, 'end_time': 1753206034251725, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-07-22 17:40:35,314 - DEBUG - Posting: http://10.0.0.254:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1753205817606057, 'end_time': 1753206035314069, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'server'} 2025-07-22 17:40:36,335 - DEBUG - [{'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'sample_count': 4}] 2025-07-22 17:40:36,821 - INFO - Deleted policy policy1 2025-07-22 17:40:36,984 - INFO - Setting flow export rate: None 2025-07-22 17:40:38,278 - DEBUG - Skipping xmpp flap check 2025-07-22 17:40:38,278 - INFO - END TEST : test_verify_session_sampling_teardown : PASSED[0:03:46] 2025-07-22 17:40:38,278 - INFO - -------------------------------------------------------------------------------- 2025-07-22 17:40:38,282 - INFO - ================================================================================ 2025-07-22 17:40:38,282 - INFO - STARTING TEST : test_verify_session_series_table_inter_vn 2025-07-22 17:40:38,282 - INFO - TEST DESCRIPTION : Verify session series table ,generated stats between different vns 1.query client session samples 2.query for server ports 3.sort results by server_port column 4.verify granularity with T=10 5.verify sampled bytes 6.verify logged bytes 7.query and verify filter by action 2025-07-22 17:40:39,546 - DEBUG - Skipping xmpp flap check 2025-07-22 17:40:39,546 - INFO - Initial checks done. Running the testcase now 2025-07-22 17:40:39,546 - INFO - 2025-07-22 17:40:39,760 - INFO - Setting flow export rate: 100 2025-07-22 17:40:39,808 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = udp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-07-22 17:40:40,206 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'c968cbcd-a3dc-48ac-a21f-86caa3f01efa', 'name': 'ctest-vn1-76443075', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['50a6e787-05af-4183-a0ed-9bd86597803b'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn1-76443075'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '207.35.205.128/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:40:40,472 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '52b2687d-c79c-48bf-a30a-03f8278977cf', 'name': 'ctest-vn2-59754775', 'tenant_id': '800f200af78c4ee6bd7febd6bb3d970c', 'project_id': '800f200af78c4ee6bd7febd6bb3d970c', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['9064865b-ffb1-4a7a-8b3f-d6883e01b5e5'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'ctest-vn2-59754775'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '206.35.68.192/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:40:40,584 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:40,642 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:40,685 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:40,784 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:40,802 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:40,814 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:40,826 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:40,837 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:40:40,846 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:40,846 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:40,859 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:40,885 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:40,885 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:40:40,885 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:40,898 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:40,909 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:40,920 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:40:40,936 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.784636'} 2025-07-22 17:40:40,943 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:40,950 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.788930'} 2025-07-22 17:40:40,959 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:40,966 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.815603'} 2025-07-22 17:40:40,974 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:40,974 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:40:40,974 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:40:40,974 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:40,985 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:41,002 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:41,113 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:41,127 - DEBUG - ==>Verifying data for policy with id: 37c3c4a3-4cd1-4f85-91f2-e48a0eeb0e58, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:40:41,127 - INFO - VN ctest-vn1-76443075 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:40:41,127 - DEBUG - Verifying the vn in opserver 2025-07-22 17:40:41,127 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:40:41,127 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:40:41,142 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:40:41,142 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:40:41,142 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:40:41,201 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:40:41,201 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:40:41,201 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:40:41,261 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:40:41,261 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:40:41,274 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:40:41,274 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:41,281 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:41,296 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:41,416 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:41,439 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:41,455 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:41,469 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:41,482 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:40:41,494 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:41,494 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:41,510 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:41,537 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:41,537 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:40:41,537 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:41,555 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:41,573 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:41,588 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:40:41,606 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.178514'} 2025-07-22 17:40:41,616 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:41,623 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.195344'} 2025-07-22 17:40:41,631 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:41,639 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.213783'} 2025-07-22 17:40:41,652 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:41,652 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:40:41,652 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:40:41,652 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:41,660 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:41,675 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:41,778 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:41,792 - DEBUG - ==>Verifying data for policy with id: 37c3c4a3-4cd1-4f85-91f2-e48a0eeb0e58, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:40:41,792 - INFO - VN ctest-vn2-59754775 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:40:41,792 - DEBUG - Verifying the vn in opserver 2025-07-22 17:40:41,792 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:40:41,793 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:40:41,808 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:40:41,808 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:40:41,808 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:40:41,869 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:40:41,869 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:40:41,869 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:40:41,929 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:40:41,929 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:40:41,944 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2', '10.0.0.27': '3'} 2025-07-22 17:40:41,944 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:41,952 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:41,965 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:42,068 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:42,085 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:42,101 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:42,112 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:42,122 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:40:42,131 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:42,131 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:42,143 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:42,163 - INFO - Verified VN network id 12 for VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:42,163 - INFO - Verifications in API Server for VN ctest-vn1-76443075 passed 2025-07-22 17:40:42,163 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:42,177 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:42,188 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:40:42,199 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/eef622bf-49e1-4af7-8a2c-955f2c04bf3e 2025-07-22 17:40:42,214 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.063249'} 2025-07-22 17:40:42,221 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:42,227 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.065727'} 2025-07-22 17:40:42,235 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:42,242 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14513073883279280300', 'uuid-lslong': '11682204163314294522'}, 'enable': 'true', 'created': '2025-07-22T17:34:38', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.091886'} 2025-07-22 17:40:42,250 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-22 17:40:42,250 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-76443075 passed 2025-07-22 17:40:42,250 - DEBUG - ====Verifying policy data for ctest-vn1-76443075 in API_Server ====== 2025-07-22 17:40:42,250 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:42,257 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:42,271 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:42,364 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/c968cbcd-a3dc-48ac-a21f-86caa3f01efa 2025-07-22 17:40:42,386 - DEBUG - ==>Verifying data for policy with id: 37c3c4a3-4cd1-4f85-91f2-e48a0eeb0e58, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:40:42,386 - INFO - VN ctest-vn1-76443075 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:40:42,386 - DEBUG - Verifying the vn in opserver 2025-07-22 17:40:42,386 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.254 2025-07-22 17:40:42,386 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:40:42,402 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:40:42,402 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.50 2025-07-22 17:40:42,402 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:40:42,462 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:40:42,462 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 virtual network link through opserver 10.0.0.23 2025-07-22 17:40:42,462 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:40:42,478 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075?flat'} 2025-07-22 17:40:42,478 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 is found in opserver 2025-07-22 17:40:42,491 - DEBUG - VRF ids for VN ctest-vn1-76443075: {'10.0.0.27': '2'} 2025-07-22 17:40:42,491 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:42,498 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:42,513 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:42,625 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:42,639 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:42,662 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:42,671 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:42,682 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:40:42,693 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:42,693 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:42,705 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:42,725 - INFO - Verified VN network id 13 for VN 52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:42,726 - INFO - Verifications in API Server for VN ctest-vn2-59754775 passed 2025-07-22 17:40:42,726 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:42,740 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:42,749 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:40:42,759 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/4bf1053a-af32-45b3-a1f6-681de9bed56f 2025-07-22 17:40:42,777 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.346954'} 2025-07-22 17:40:42,790 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:42,798 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.369539'} 2025-07-22 17:40:42,806 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:42,812 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5958940146393696447', 'uuid-lslong': '11748206942299453391'}, 'enable': 'true', 'created': '2025-07-22T17:35:18', 'last-modified': '2025-07-22T17:40:40', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.386904'} 2025-07-22 17:40:42,821 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-22 17:40:42,822 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-59754775 passed 2025-07-22 17:40:42,822 - DEBUG - ====Verifying policy data for ctest-vn2-59754775 in API_Server ====== 2025-07-22 17:40:42,822 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:42,830 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:42,844 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:42,950 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/52b2687d-c79c-48bf-a30a-03f8278977cf 2025-07-22 17:40:42,962 - DEBUG - ==>Verifying data for policy with id: 37c3c4a3-4cd1-4f85-91f2-e48a0eeb0e58, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-73069084', 'policy1'] 2025-07-22 17:40:42,962 - INFO - VN ctest-vn2-59754775 Policy verification: verify_vn_policy_in_api_server, status: True 2025-07-22 17:40:42,962 - DEBUG - Verifying the vn in opserver 2025-07-22 17:40:42,962 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.254 2025-07-22 17:40:42,962 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:40:42,977 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:40:42,977 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.50 2025-07-22 17:40:42,977 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:40:42,991 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:40:42,992 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 virtual network link through opserver 10.0.0.23 2025-07-22 17:40:42,992 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:40:43,006 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775?flat'} 2025-07-22 17:40:43,006 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 is found in opserver 2025-07-22 17:40:43,022 - DEBUG - VRF ids for VN ctest-vn2-59754775: {'10.0.0.19': '2', '10.0.0.27': '3'} 2025-07-22 17:40:43,022 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:43,030 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:43,044 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:43,153 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:40:43,169 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:40:43,180 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:40:43,193 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:40:43,202 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:40:43,214 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:40:43,214 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:40:43,227 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:40:43,245 - INFO - Verified VN network id 14 for VN abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:40:43,246 - INFO - Verifications in API Server for VN ctest-fip_vn-19357511 passed 2025-07-22 17:40:43,246 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:40:43,257 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:40:43,266 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:40:43,275 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/8379c185-f643-4e6b-b304-10ca018bc057 2025-07-22 17:40:43,292 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:23.897158'} 2025-07-22 17:40:43,301 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:40:43,308 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:23.914444'} 2025-07-22 17:40:43,316 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:40:43,323 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-73069084', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12376857544748712399', 'uuid-lslong': '13681745999049984447'}, 'enable': 'true', 'created': '2025-07-22T17:35:19', 'last-modified': '2025-07-22T17:35:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:23.934870'} 2025-07-22 17:40:43,331 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-22 17:40:43,331 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-19357511 passed 2025-07-22 17:40:43,331 - DEBUG - ====Verifying policy data for ctest-fip_vn-19357511 in API_Server ====== 2025-07-22 17:40:43,331 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:43,339 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:43,356 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:40:43,461 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/abc36e5c-8e5c-41cf-bddf-537f447f09bf 2025-07-22 17:40:43,476 - DEBUG - =>VN ctest-fip_vn-19357511 has no policy to be verified 2025-07-22 17:40:43,476 - DEBUG - Verifying the vn in opserver 2025-07-22 17:40:43,476 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.254 2025-07-22 17:40:43,476 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:40:43,492 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:40:43,492 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.50 2025-07-22 17:40:43,492 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:40:43,508 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:40:43,508 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 virtual network link through opserver 10.0.0.23 2025-07-22 17:40:43,508 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:40:43,524 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511?flat'} 2025-07-22 17:40:43,524 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 is found in opserver 2025-07-22 17:40:43,539 - DEBUG - VRF ids for VN ctest-fip_vn-19357511: {'10.0.0.19': '3'} 2025-07-22 17:40:43,540 - INFO - start time= 1753206043540066 2025-07-22 17:40:43,540 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:43,540 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:43,540 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 206.35.68.195 2025-07-22 17:40:43,540 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-84203418.pid --stats_file /tmp/server-ctest-random-84203418.stats 0<&- &> /tmp/server-ctest-random-84203418.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:40:43,540 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-84203418.pid --stats_file /tmp/server-ctest-random-84203418.stats 0<&- &> /tmp/server-ctest-random-84203418.log & 2025-07-22 17:40:43,760 - DEBUG - None 2025-07-22 17:40:43,761 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:43,761 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 207.35.205.131 2025-07-22 17:40:43,761 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-84203418.pid --stats_file /tmp/client-ctest-random-84203418.stats 0<&- &> /tmp/client-ctest-random-84203418.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:40:43,761 - DEBUG - nohup python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-84203418.pid --stats_file /tmp/client-ctest-random-84203418.stats 0<&- &> /tmp/client-ctest-random-84203418.log & 2025-07-22 17:40:43,985 - DEBUG - None 2025-07-22 17:40:43,985 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:43,986 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:43,986 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 206.35.68.195 2025-07-22 17:40:43,986 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-76510189.pid --stats_file /tmp/server-ctest-random-76510189.stats 0<&- &> /tmp/server-ctest-random-76510189.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:40:43,986 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-76510189.pid --stats_file /tmp/server-ctest-random-76510189.stats 0<&- &> /tmp/server-ctest-random-76510189.log & 2025-07-22 17:40:44,213 - DEBUG - None 2025-07-22 17:40:44,213 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:44,213 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 207.35.205.131 2025-07-22 17:40:44,214 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-76510189.pid --stats_file /tmp/client-ctest-random-76510189.stats 0<&- &> /tmp/client-ctest-random-76510189.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:40:44,214 - DEBUG - nohup python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-76510189.pid --stats_file /tmp/client-ctest-random-76510189.stats 0<&- &> /tmp/client-ctest-random-76510189.log & 2025-07-22 17:40:44,438 - DEBUG - None 2025-07-22 17:40:44,438 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:44,439 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:44,439 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 206.35.68.195 2025-07-22 17:40:44,439 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-75533717.pid --stats_file /tmp/server-ctest-random-75533717.stats 0<&- &> /tmp/server-ctest-random-75533717.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:40:44,439 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-75533717.pid --stats_file /tmp/server-ctest-random-75533717.stats 0<&- &> /tmp/server-ctest-random-75533717.log & 2025-07-22 17:40:44,667 - DEBUG - None 2025-07-22 17:40:44,667 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-07-22 17:40:44,667 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 207.35.205.131 2025-07-22 17:40:44,667 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-75533717.pid --stats_file /tmp/client-ctest-random-75533717.stats 0<&- &> /tmp/client-ctest-random-75533717.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:40:44,667 - DEBUG - nohup python /tmp/udpechoclient.py --servers 206.35.68.195 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-75533717.pid --stats_file /tmp/client-ctest-random-75533717.stats 0<&- &> /tmp/client-ctest-random-75533717.log & 2025-07-22 17:40:44,909 - DEBUG - None 2025-07-22 17:40:54,910 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-84203418.pid); sync; cat /tmp/client-ctest-random-84203418.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.27, gateway password: c0ntrail123 2025-07-22 17:40:54,910 - DEBUG - kill $(cat /tmp/client-ctest-random-84203418.pid); sync; cat /tmp/client-ctest-random-84203418.stats 2025-07-22 17:40:55,234 - DEBUG - cat: can't open '/tmp/client-ctest-random-84203418.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/client-ctest-random-84203418.stats': No such file or directory 2025-07-22 17:40:55,234 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-84203418.pid); sync; cat /tmp/server-ctest-random-84203418.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-07-22 17:40:55,234 - DEBUG - kill $(cat /tmp/server-ctest-random-84203418.pid); sync; cat /tmp/server-ctest-random-84203418.stats 2025-07-22 17:40:55,557 - DEBUG - cat: can't open '/tmp/server-ctest-random-84203418.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/server-ctest-random-84203418.stats': No such file or directory 2025-07-22 17:40:55,557 - INFO - Client - Sent: 0, Received: 0 2025-07-22 17:40:55,557 - INFO - Server - Sent: 0, Received: 0 2025-07-22 17:40:56,062 - INFO - Deleted policy policy1 2025-07-22 17:40:56,232 - INFO - Setting flow export rate: None 2025-07-22 17:40:57,489 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Tue Jul 22 17:40:55 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/analytics/test_analytics_resource.py in test_verify_session_series_table_inter_vn(self=) 1336 start_time = self.analytics_obj.getstarttime(vm_node_ip) 1337 self.logger.info("start time= %s" % (start_time)) 1338 self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) 1339 src_vn = self.res.vn1_fixture.vn_fq_name 1340 dst_vn = self.res.vn2_fixture.vn_fq_name self = self.setup_and_create_streams = > self.res = self.res.vn1_vm1_fixture = self.res.vn2_vm2_fixture = /contrail-test/serial_scripts/analytics/base.py in setup_and_create_streams(self=, src_vm=, dst_vm=, sport=8000, dport=9003, count=100) 168 time.sleep(10) 169 for traffic_obj in traffic_objs: 170 self.stop_traffic(traffic_obj) 171 172 #end setup_create_streams self = self.stop_traffic = > traffic_obj = /contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False) 796 assert False, "No drops observed in packets %s"%msg 797 elif expectation: 798 assert sent and recv, "Packets not %s"%msg 799 if recv*100/float(sent) < 90: 800 assert False, "Packets not %s"%msg sent = 0 recv = 0 msg = 'transferred between 207.35.205.131 and 206.35.68.195, proto udp sport 8000 and dport 9000' AssertionError: Packets not transferred between 207.35.205.131 and 206.35.68.195, proto udp sport 8000 and dport 9000 __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('Packets not transferred between 207.35.205.131 a...06.35.68.195, proto udp sport 8000 and dport 9000',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/analytics/test_analytics_resource.py", line 1338, in test_verify_session_series_table_inter_vn self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) File "/contrail-test/serial_scripts/analytics/base.py", line 170, in setup_and_create_streams self.stop_traffic(traffic_obj) File "/contrail-test/common/base.py", line 798, in stop_traffic assert sent and recv, "Packets not %s"%msg AssertionError: Packets not transferred between 207.35.205.131 and 206.35.68.195, proto udp sport 8000 and dport 9000 2025-07-22 17:40:57,490 - DEBUG - Skipping xmpp flap check 2025-07-22 17:40:57,490 - INFO - 2025-07-22 17:40:57,490 - INFO - END TEST : test_verify_session_series_table_inter_vn : FAILED[0:00:19] 2025-07-22 17:40:57,490 - INFO - -------------------------------------------------------------------------------- 2025-07-22 17:40:57,492 - INFO - Deleting VM mltf_vm 2025-07-22 17:40:57,559 - INFO - Deleting VM ctest-fvn_vm1-13112516 2025-07-22 17:40:57,634 - INFO - Deleting VM ctest-vn2_vm2-24845532 2025-07-22 17:40:57,707 - INFO - Deleting VN ctest-fip_vn-19357511 2025-07-22 17:40:57,750 - DEBUG - VN abc36e5c-8e5c-41cf-bddf-537f447f09bf still in use: Unable to complete operation on network abc36e5c-8e5c-41cf-bddf-537f447f09bf. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-49c3ed47-0439-484e-a6ce-f882b4aea6a1'] 2025-07-22 17:40:57,750 - WARNING - Deleting VN ctest-fip_vn-19357511 failed..Will retry 2025-07-22 17:40:59,930 - DEBUG - Response for deleting network () 2025-07-22 17:40:59,931 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/4f997c96-03bc-40fa-bf66-180df9872fde 2025-07-22 17:40:59,945 - DEBUG - Response Code: 404 2025-07-22 17:40:59,945 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:40:59,956 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:40:59,977 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:41:00,098 - INFO - Validated that VN ctest-fip_vn-19357511 is not found in API Server 2025-07-22 17:41:00,111 - DEBUG - VN ctest-fip_vn-19357511 is not present in Agent 10.0.0.19 2025-07-22 17:41:00,123 - DEBUG - VN ctest-fip_vn-19357511 is not present in Agent 10.0.0.27 2025-07-22 17:41:00,123 - INFO - Validated that VN ctest-fip_vn-19357511 is not in any agent 2025-07-22 17:41:00,129 - DEBUG - VRF 3 is not seen in agent 10.0.0.19 2025-07-22 17:41:00,136 - DEBUG - Vrouter 10.0.0.19 does not have vrf 3 for VN ctest-fip_vn-19357511 2025-07-22 17:41:00,136 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-fip_vn-19357511 2025-07-22 17:41:00,164 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-fip_vn-19357511 info 2025-07-22 17:41:00,164 - INFO - Deleting VN ctest-vn2-59754775 2025-07-22 17:41:00,603 - DEBUG - Response for deleting network () 2025-07-22 17:41:00,603 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/e5b1b478-0117-4e3e-9239-27caf44cb043 2025-07-22 17:41:00,610 - DEBUG - Response Code: 404 2025-07-22 17:41:00,610 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:41:00,616 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:41:00,629 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:41:00,749 - INFO - Validated that VN ctest-vn2-59754775 is not found in API Server 2025-07-22 17:41:00,763 - DEBUG - VN ctest-vn2-59754775 is not present in Agent 10.0.0.19 2025-07-22 17:41:00,776 - DEBUG - VN ctest-vn2-59754775 is not present in Agent 10.0.0.27 2025-07-22 17:41:00,776 - INFO - Validated that VN ctest-vn2-59754775 is not in any agent 2025-07-22 17:41:00,785 - DEBUG - VRF 2 is not seen in agent 10.0.0.19 2025-07-22 17:41:00,793 - DEBUG - Vrouter 10.0.0.19 does not have vrf 2 for VN ctest-vn2-59754775 2025-07-22 17:41:00,800 - DEBUG - VRF 3 is not seen in agent 10.0.0.27 2025-07-22 17:41:00,807 - DEBUG - Vrouter 10.0.0.27 does not have vrf 3 for VN ctest-vn2-59754775 2025-07-22 17:41:00,807 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn2-59754775 2025-07-22 17:41:00,832 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn2-59754775 info 2025-07-22 17:41:00,833 - INFO - Deleting VM ctest-vn1_vm2-64726636 2025-07-22 17:41:00,912 - INFO - Deleting VM ctest-vn1_vm1-61388391 2025-07-22 17:41:00,981 - INFO - Deleting VN ctest-vn1-76443075 2025-07-22 17:41:01,047 - DEBUG - VN c968cbcd-a3dc-48ac-a21f-86caa3f01efa still in use: Unable to complete operation on network c968cbcd-a3dc-48ac-a21f-86caa3f01efa. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-7de0defb-71c1-4f92-b420-b98bc355356e'] 2025-07-22 17:41:01,047 - WARNING - Deleting VN ctest-vn1-76443075 failed..Will retry 2025-07-22 17:41:03,222 - DEBUG - Response for deleting network () 2025-07-22 17:41:03,222 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/dc872179-d2f9-4dcb-8c53-4420559747ad 2025-07-22 17:41:03,228 - DEBUG - Response Code: 404 2025-07-22 17:41:03,229 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:41:03,245 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:41:03,261 - DEBUG - Requesting: http://10.0.0.254:8082/project/800f200a-f78c-4ee6-bd7f-ebd6bb3d970c 2025-07-22 17:41:03,351 - INFO - Validated that VN ctest-vn1-76443075 is not found in API Server 2025-07-22 17:41:03,364 - DEBUG - VN ctest-vn1-76443075 is not present in Agent 10.0.0.19 2025-07-22 17:41:03,376 - DEBUG - VN ctest-vn1-76443075 is not present in Agent 10.0.0.27 2025-07-22 17:41:03,376 - INFO - Validated that VN ctest-vn1-76443075 is not in any agent 2025-07-22 17:41:03,382 - DEBUG - VRF 2 is not seen in agent 10.0.0.27 2025-07-22 17:41:03,389 - DEBUG - Vrouter 10.0.0.27 does not have vrf 2 for VN ctest-vn1-76443075 2025-07-22 17:41:03,389 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-73069084:ctest-vn1-76443075 2025-07-22 17:41:03,415 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn1-76443075 info 2025-07-22 17:41:04,192 - INFO - Deleted project: ctest-AnalyticsTestSanityWithResource-73069084, ID : 800f200a-f78c-4ee6-bd7f-ebd6bb3d970c