2025-08-03 00:32:23,786 - INFO - Domain Default found not creating 2025-08-03 00:32:23,954 - INFO - Project ctest-AnalyticsTestSanityWithResource-81719282 not found, creating it 2025-08-03 00:32:24,623 - INFO - Created Project:ctest-AnalyticsTestSanityWithResource-81719282, ID : b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:32:26,996 - DEBUG - Response for create_network : {'network': {'id': '175375e7-c1ac-461b-9825-88fae2d5f99d', 'name': 'ctest-vn1-24950719', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn1-24950719'], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:32:27,389 - DEBUG - Response for create_subnet : {'subnet': {'id': 'b4152e6b-0e81-466c-85fe-9face7e7259f', 'name': '', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'network_id': '175375e7-c1ac-461b-9825-88fae2d5f99d', 'ip_version': 4, 'cidr': '50.140.35.64/26', 'allocation_pools': [{'start': '50.140.35.66', 'end': '50.140.35.126'}], 'gateway_ip': '50.140.35.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '50.140.35.66', 'tags': [], 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0'}} 2025-08-03 00:32:27,416 - INFO - Created VN ctest-vn1-24950719 2025-08-03 00:32:27,432 - DEBUG - VN ctest-vn1-24950719 UUID is 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:27,537 - DEBUG - Services list from nova: [, , , ] 2025-08-03 00:32:27,538 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3743-2', 'an-jenkins-deploy-platform-ansible-os-3743-1']} 2025-08-03 00:32:28,818 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3743-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3743-2) 2025-08-03 00:32:30,315 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3743-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3743-2) 2025-08-03 00:32:30,315 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:32:30,921 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:32:31,008 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:32:31,034 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:32:31,153 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:31,187 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:31,213 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:32:31,229 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:32:31,244 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:32:31,260 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:32:31,260 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:31,298 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:32:31,320 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:31,321 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:32:31,321 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:31,333 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:32:31,353 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:32:31,375 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:32:31,415 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:32:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:04.110972'} 2025-08-03 00:32:31,426 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:32:31,435 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:32:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:04.117609'} 2025-08-03 00:32:31,450 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:32:31,459 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:32:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:04.160348'} 2025-08-03 00:32:31,473 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:32:31,473 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:32:31,473 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:32:31,473 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:32:31,485 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:32:31,502 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:32:31,649 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:32:31,663 - DEBUG - =>VN ctest-vn1-24950719 has no policy to be verified 2025-08-03 00:32:31,663 - DEBUG - Verifying the vn in opserver 2025-08-03 00:32:31,663 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:32:31,663 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:32:31,973 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:32:32,042 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:32:32,042 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:32:32,042 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:32:32,337 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:32:32,422 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:32:32,422 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:32:32,422 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:32:32,726 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:32:32,818 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:32:32,818 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:32:32,830 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:32:32,831 - INFO - Waiting for VM ctest-vn1_vm1-71948368 to be up.. 2025-08-03 00:32:32,955 - DEBUG - VM is in ACTIVE state now 2025-08-03 00:32:32,955 - INFO - VM name : ctest-vn1_vm1-71948368 2025-08-03 00:32:33,074 - DEBUG - VM ctest-vn1_vm1-71948368 ID is 9756dfd2-ce65-487e-9ee5-ce41be79c2d5 2025-08-03 00:32:33,110 - DEBUG - VM ctest-vn1_vm1-71948368 launched on Node an-jenkins-deploy-platform-ansible-os-3743-2 2025-08-03 00:32:33,217 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine/9756dfd2-ce65-487e-9ee5-ce41be79c2d5 2025-08-03 00:32:33,514 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine/9756dfd2-ce65-487e-9ee5-ce41be79c2d5 2025-08-03 00:32:33,566 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine-interface/b74ca673-62cb-49a0-a773-f3dafad34ad1 2025-08-03 00:32:36,808 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1002ms') 2025-08-03 00:32:36,808 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-71948368 failed! 2025-08-03 00:32:36,845 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is 50.140.35.65 and allocation pool is NOT set 2025-08-03 00:32:40,938 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms') 2025-08-03 00:32:40,938 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-71948368 failed! 2025-08-03 00:32:40,971 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is 50.140.35.65 and allocation pool is NOT set 2025-08-03 00:32:45,059 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1028ms') 2025-08-03 00:32:45,060 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-71948368 failed! 2025-08-03 00:32:45,077 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is 50.140.35.65 and allocation pool is NOT set 2025-08-03 00:32:49,161 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-08-03 00:32:49,161 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-71948368 failed! 2025-08-03 00:32:49,183 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is 50.140.35.65 and allocation pool is NOT set 2025-08-03 00:32:51,258 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=8.91 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.88 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 3.880/6.395/8.911/2.515 ms') 2025-08-03 00:32:51,258 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn1_vm1-71948368 passed 2025-08-03 00:32:51,377 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:32:51,378 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-71948368, IP 50.140.35.67, Port 22 2025-08-03 00:32:51,443 - DEBUG - Error on ssh to cirros@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-03 00:32:51,563 - DEBUG - VM ctest-vn1_vm1-71948368 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-03 00:32:56,565 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:32:56,565 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-71948368, IP 50.140.35.67, Port 22 2025-08-03 00:32:56,632 - DEBUG - Error on ssh to cirros@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-03 00:32:56,735 - DEBUG - VM ctest-vn1_vm1-71948368 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-03 00:33:01,736 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:01,736 - DEBUG - Waiting to SSH to VM ctest-vn1_vm1-71948368, IP 50.140.35.67, Port 22 2025-08-03 00:33:01,904 - DEBUG - VM ctest-vn1_vm1-71948368 is ready for SSH connections 2025-08-03 00:33:01,904 - INFO - Waiting for VM ctest-vn1_vm2-57667192 to be up.. 2025-08-03 00:33:02,027 - DEBUG - VM is in ACTIVE state now 2025-08-03 00:33:02,028 - INFO - VM name : ctest-vn1_vm2-57667192 2025-08-03 00:33:02,134 - DEBUG - VM ctest-vn1_vm2-57667192 ID is a997937a-9e2c-4a9e-ac60-a54418d45376 2025-08-03 00:33:02,134 - DEBUG - VM ctest-vn1_vm2-57667192 launched on Node an-jenkins-deploy-platform-ansible-os-3743-2 2025-08-03 00:33:02,240 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine/a997937a-9e2c-4a9e-ac60-a54418d45376 2025-08-03 00:33:02,257 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine-interface/c1b6ef7f-4392-49c1-b235-3ba847a09db0 2025-08-03 00:33:03,500 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=7.85 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.546 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.546/4.198/7.851/3.652 ms') 2025-08-03 00:33:03,500 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-vn1_vm2-57667192 passed 2025-08-03 00:33:03,646 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:03,646 - DEBUG - Waiting to SSH to VM ctest-vn1_vm2-57667192, IP 50.140.35.68, Port 22 2025-08-03 00:33:03,816 - DEBUG - VM ctest-vn1_vm2-57667192 is ready for SSH connections 2025-08-03 00:33:04,241 - DEBUG - Response for create_network : {'network': {'id': 'aa59a538-60b9-42f1-b128-ae205a1a5f7d', 'name': 'ctest-vn2-90001139', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn2-90001139'], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:33:04,533 - DEBUG - Response for create_subnet : {'subnet': {'id': 'ddafc3f1-5753-4d48-929e-fbcde432ab72', 'name': '', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'network_id': 'aa59a538-60b9-42f1-b128-ae205a1a5f7d', 'ip_version': 4, 'cidr': '53.96.17.128/26', 'allocation_pools': [{'start': '53.96.17.130', 'end': '53.96.17.190'}], 'gateway_ip': '53.96.17.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '53.96.17.130', 'tags': [], 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0'}} 2025-08-03 00:33:04,556 - INFO - Created VN ctest-vn2-90001139 2025-08-03 00:33:04,573 - DEBUG - VN ctest-vn2-90001139 UUID is aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:05,091 - DEBUG - Response for create_network : {'network': {'id': '57f73b3a-deee-4318-b700-39049cc9da49', 'name': 'ctest-fip_vn-71509828', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-fip_vn-71509828'], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:33:05,366 - DEBUG - Response for create_subnet : {'subnet': {'id': '886d7335-c564-4193-80b7-7da774d875af', 'name': '', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'network_id': '57f73b3a-deee-4318-b700-39049cc9da49', 'ip_version': 4, 'cidr': '34.143.122.64/26', 'allocation_pools': [{'start': '34.143.122.66', 'end': '34.143.122.126'}], 'gateway_ip': '34.143.122.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '34.143.122.66', 'tags': [], 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0'}} 2025-08-03 00:33:05,388 - INFO - Created VN ctest-fip_vn-71509828 2025-08-03 00:33:05,401 - DEBUG - VN ctest-fip_vn-71509828 UUID is 57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:05,501 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3743-2', 'an-jenkins-deploy-platform-ansible-os-3743-1']} 2025-08-03 00:33:06,872 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3743-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3743-1) 2025-08-03 00:33:08,301 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3743-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3743-1) 2025-08-03 00:33:10,034 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3743-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3743-2) 2025-08-03 00:33:10,035 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:10,044 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:10,065 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:10,201 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:10,216 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:10,235 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:10,245 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:10,255 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:33:10,263 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:10,263 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:10,278 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:10,310 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:10,310 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:33:10,311 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:10,323 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:10,338 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:10,357 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:33:10,373 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:32:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:43.071236'} 2025-08-03 00:33:10,381 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:10,394 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:32:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:43.075841'} 2025-08-03 00:33:10,414 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:10,427 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:32:27', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:43.129804'} 2025-08-03 00:33:10,467 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:10,467 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:33:10,467 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:33:10,467 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:10,476 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:10,511 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:10,641 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:10,671 - DEBUG - =>VN ctest-vn1-24950719 has no policy to be verified 2025-08-03 00:33:10,671 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:10,671 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:10,671 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:10,765 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:10,765 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:10,765 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:10,885 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:10,885 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:10,885 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:10,983 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:10,983 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:33:11,006 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:33:11,006 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:11,025 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:11,047 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:11,171 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:11,191 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:11,217 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:11,229 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:11,249 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:33:11,259 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:11,259 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:11,272 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:11,303 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:11,303 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:33:11,303 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:11,320 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:11,330 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:11,341 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:33:11,361 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:04', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.881920'} 2025-08-03 00:33:11,370 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:11,391 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:04', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.921529'} 2025-08-03 00:33:11,406 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:11,414 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:04', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.936747'} 2025-08-03 00:33:11,437 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:11,437 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:33:11,438 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:33:11,438 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:11,454 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:11,474 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:11,579 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:11,597 - DEBUG - =>VN ctest-vn2-90001139 has no policy to be verified 2025-08-03 00:33:11,597 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:11,597 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:11,597 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:11,673 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:11,674 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:11,674 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:11,763 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:11,763 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:11,764 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:11,851 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:11,851 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:33:11,864 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2'} 2025-08-03 00:33:11,864 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:11,872 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:11,888 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:12,016 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:12,039 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:12,055 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:12,067 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:12,077 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:33:12,091 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:12,091 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:12,118 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:12,154 - INFO - Verified VN network id 14 for VN 57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:12,155 - INFO - Verifications in API Server for VN ctest-fip_vn-71509828 passed 2025-08-03 00:33:12,156 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:12,169 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:12,185 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:12,198 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:33:12,230 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.963599'} 2025-08-03 00:33:12,243 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:12,258 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:06.997346'} 2025-08-03 00:33:12,277 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:12,293 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:07.031093'} 2025-08-03 00:33:12,309 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:12,310 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-71509828 passed 2025-08-03 00:33:12,310 - DEBUG - ====Verifying policy data for ctest-fip_vn-71509828 in API_Server ====== 2025-08-03 00:33:12,310 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:12,320 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:12,338 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:12,453 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:12,466 - DEBUG - =>VN ctest-fip_vn-71509828 has no policy to be verified 2025-08-03 00:33:12,466 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:12,466 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:12,467 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:12,482 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:33:12,482 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:12,483 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:12,509 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:33:12,509 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:12,509 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:12,526 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:33:12,526 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is found in opserver 2025-08-03 00:33:12,541 - DEBUG - VRF ids for VN ctest-fip_vn-71509828: {'10.0.0.48': '3'} 2025-08-03 00:33:12,541 - INFO - Waiting for VM ctest-fvn_vm1-96269498 to be up.. 2025-08-03 00:33:12,649 - DEBUG - VM is in ACTIVE state now 2025-08-03 00:33:12,650 - INFO - VM name : ctest-fvn_vm1-96269498 2025-08-03 00:33:12,781 - DEBUG - VM ctest-fvn_vm1-96269498 ID is c5a74585-9792-4dcc-bc36-ddbc7e3b2b53 2025-08-03 00:33:12,781 - DEBUG - VM ctest-fvn_vm1-96269498 launched on Node an-jenkins-deploy-platform-ansible-os-3743-1 2025-08-03 00:33:12,930 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine/c5a74585-9792-4dcc-bc36-ddbc7e3b2b53 2025-08-03 00:33:12,947 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine-interface/83b900f1-640e-4014-86f2-73525f077cad 2025-08-03 00:33:16,193 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-08-03 00:33:16,193 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-96269498 failed! 2025-08-03 00:33:16,230 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is 34.143.122.65 and allocation pool is NOT set 2025-08-03 00:33:20,321 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms') 2025-08-03 00:33:20,321 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-96269498 failed! 2025-08-03 00:33:20,353 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is 34.143.122.65 and allocation pool is NOT set 2025-08-03 00:33:24,424 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms') 2025-08-03 00:33:24,424 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-96269498 failed! 2025-08-03 00:33:24,457 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is 34.143.122.65 and allocation pool is NOT set 2025-08-03 00:33:28,553 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms') 2025-08-03 00:33:28,554 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-96269498 failed! 2025-08-03 00:33:28,574 - DEBUG - Gateway for vn default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is 34.143.122.65 and allocation pool is NOT set 2025-08-03 00:33:30,638 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=8.99 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=2.25 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.253/5.620/8.987/3.367 ms') 2025-08-03 00:33:30,638 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-fvn_vm1-96269498 passed 2025-08-03 00:33:30,788 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:30,788 - DEBUG - Waiting to SSH to VM ctest-fvn_vm1-96269498, IP 34.143.122.67, Port 22 2025-08-03 00:33:30,854 - DEBUG - Error on ssh to cirros@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-03 00:33:30,971 - DEBUG - VM ctest-fvn_vm1-96269498 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-03 00:33:35,972 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:35,972 - DEBUG - Waiting to SSH to VM ctest-fvn_vm1-96269498, IP 34.143.122.67, Port 22 2025-08-03 00:33:36,038 - DEBUG - Error on ssh to cirros@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-08-03 00:33:36,141 - DEBUG - VM ctest-fvn_vm1-96269498 is NOT ready for SSH connections, VM status: ACTIVE 2025-08-03 00:33:41,142 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:41,142 - DEBUG - Waiting to SSH to VM ctest-fvn_vm1-96269498, IP 34.143.122.67, Port 22 2025-08-03 00:33:41,299 - DEBUG - VM ctest-fvn_vm1-96269498 is ready for SSH connections 2025-08-03 00:33:41,300 - INFO - Waiting for VM ctest-vn2_vm2-50961308 to be up.. 2025-08-03 00:33:41,409 - DEBUG - VM is in ACTIVE state now 2025-08-03 00:33:41,410 - INFO - VM name : ctest-vn2_vm2-50961308 2025-08-03 00:33:41,514 - DEBUG - VM ctest-vn2_vm2-50961308 ID is 76b938ae-8fee-4e41-b2e0-b1cc342b89d3 2025-08-03 00:33:41,514 - DEBUG - VM ctest-vn2_vm2-50961308 launched on Node an-jenkins-deploy-platform-ansible-os-3743-1 2025-08-03 00:33:41,628 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine/76b938ae-8fee-4e41-b2e0-b1cc342b89d3 2025-08-03 00:33:41,649 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine-interface/233030c1-6ba2-424e-9466-ab1d28086f4a 2025-08-03 00:33:42,893 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=7.41 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.374 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.374/3.890/7.406/3.516 ms') 2025-08-03 00:33:42,893 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vn2_vm2-50961308 passed 2025-08-03 00:33:43,039 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:43,039 - DEBUG - Waiting to SSH to VM ctest-vn2_vm2-50961308, IP 53.96.17.131, Port 22 2025-08-03 00:33:43,206 - DEBUG - VM ctest-vn2_vm2-50961308 is ready for SSH connections 2025-08-03 00:33:43,206 - INFO - Waiting for VM mltf_vm to be up.. 2025-08-03 00:33:43,342 - DEBUG - VM is in ACTIVE state now 2025-08-03 00:33:43,342 - INFO - VM name : mltf_vm 2025-08-03 00:33:43,468 - DEBUG - VM mltf_vm ID is 7020187f-b93b-4b93-9062-d79c14469ecf 2025-08-03 00:33:43,468 - DEBUG - VM mltf_vm launched on Node an-jenkins-deploy-platform-ansible-os-3743-2 2025-08-03 00:33:43,705 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine/7020187f-b93b-4b93-9062-d79c14469ecf 2025-08-03 00:33:43,725 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine-interface/9de49402-89d2-4d32-aaf3-37330534f1a4 2025-08-03 00:33:43,748 - DEBUG - Requesting: http://10.0.0.50:8082/virtual-machine-interface/a964d7e7-e4e7-47c3-88d8-875348b36e18 2025-08-03 00:33:45,018 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n64 bytes from 169.254.0.5: icmp_seq=1 ttl=63 time=3.00 ms\r\n64 bytes from 169.254.0.5: icmp_seq=2 ttl=63 time=0.624 ms\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.624/1.814/3.004/1.190 ms') 2025-08-03 00:33:45,018 - INFO - Ping to Metadata IP 169.254.0.5 of VM mltf_vm passed 2025-08-03 00:33:45,148 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:45,148 - DEBUG - Waiting to SSH to VM mltf_vm, IP 50.140.35.69, Port 22 2025-08-03 00:33:45,312 - DEBUG - VM mltf_vm is ready for SSH connections 2025-08-03 00:33:45,464 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:45,464 - DEBUG - Waiting to SSH to VM mltf_vm, IP 50.140.35.69, Port 22 2025-08-03 00:33:45,636 - DEBUG - VM mltf_vm is ready for SSH connections 2025-08-03 00:33:45,638 - INFO - ================================================================================ 2025-08-03 00:33:45,638 - INFO - STARTING TEST : test_verify_session_record_table_inter_vn 2025-08-03 00:33:45,639 - INFO - TEST DESCRIPTION : Verify session record table ,generated stats between different vns 1.query and verify number of client session records 2.query and verify number of server session records 3.query with local_ip server_port protocol 4.query with server_port local_ip filter by server_port 5.query with client_port remote_ip filter by client_port Total we get three record limit by 2 6.query with sort_fields 2025-08-03 00:33:46,987 - DEBUG - Skipping xmpp flap check 2025-08-03 00:33:46,987 - INFO - Initial checks done. Running the testcase now 2025-08-03 00:33:46,987 - INFO - 2025-08-03 00:33:47,241 - INFO - Setting flow export rate: 100 2025-08-03 00:33:47,294 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = udp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-08-03 00:33:47,780 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '175375e7-c1ac-461b-9825-88fae2d5f99d', 'name': 'ctest-vn1-24950719', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['b4152e6b-0e81-466c-85fe-9face7e7259f'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn1-24950719'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '50.140.35.64/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:33:48,014 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'aa59a538-60b9-42f1-b128-ae205a1a5f7d', 'name': 'ctest-vn2-90001139', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['ddafc3f1-5753-4d48-929e-fbcde432ab72'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn2-90001139'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '53.96.17.128/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:33:48,108 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:48,118 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:48,131 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:48,240 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:48,262 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:48,280 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:48,294 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:48,328 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:33:48,349 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:48,350 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:48,381 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:48,421 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:48,421 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:33:48,421 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:48,438 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:48,452 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:48,464 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:33:48,479 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.765463'} 2025-08-03 00:33:48,487 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:48,496 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.780755'} 2025-08-03 00:33:48,505 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:48,511 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.798080'} 2025-08-03 00:33:48,519 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:48,519 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:33:48,519 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:33:48,519 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:48,526 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:48,540 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:48,647 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:48,666 - DEBUG - ==>Verifying data for policy with id: 7cb38503-bbde-4086-8a06-1f6fb55b1275, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:33:48,666 - INFO - VN ctest-vn1-24950719 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:33:48,666 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:48,666 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:48,666 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:48,681 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:48,681 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:48,681 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:48,697 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:48,697 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:48,697 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:48,713 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:48,714 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:33:48,727 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:33:48,727 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:48,735 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:48,750 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:48,864 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:48,880 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:48,891 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:48,907 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:48,918 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:33:48,926 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:48,926 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:48,940 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:48,968 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:48,968 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:33:48,968 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:48,982 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:48,991 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:49,000 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:33:49,016 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.040995'} 2025-08-03 00:33:49,027 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:49,036 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.066191'} 2025-08-03 00:33:49,045 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:49,052 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.078048'} 2025-08-03 00:33:49,061 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:49,062 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:33:49,062 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:33:49,062 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:49,071 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:49,083 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:49,171 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:49,186 - DEBUG - ==>Verifying data for policy with id: 7cb38503-bbde-4086-8a06-1f6fb55b1275, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:33:49,186 - INFO - VN ctest-vn2-90001139 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:33:49,186 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:49,186 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:49,186 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:49,202 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:49,202 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:49,202 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:49,219 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:49,219 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:49,219 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:49,233 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:49,233 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:33:49,253 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2', '10.0.0.49': '3'} 2025-08-03 00:33:49,253 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:49,261 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:49,275 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:49,386 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:49,398 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:49,411 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:49,423 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:49,433 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:33:49,442 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:49,442 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:49,454 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:49,480 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:49,480 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:33:49,480 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:49,503 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:49,519 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:33:49,533 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:33:49,551 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.834987'} 2025-08-03 00:33:49,562 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:49,573 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.854898'} 2025-08-03 00:33:49,591 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:49,603 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.887971'} 2025-08-03 00:33:49,614 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:33:49,614 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:33:49,614 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:33:49,614 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:49,622 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:49,638 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:49,734 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:33:49,745 - DEBUG - ==>Verifying data for policy with id: 7cb38503-bbde-4086-8a06-1f6fb55b1275, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:33:49,746 - INFO - VN ctest-vn1-24950719 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:33:49,746 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:49,746 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:49,746 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:49,760 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:49,760 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:49,760 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:49,775 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:49,775 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:49,775 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:49,791 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:33:49,792 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:33:49,804 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:33:49,804 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:49,811 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:49,825 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:49,936 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:49,949 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:49,962 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:49,971 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:49,985 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:33:49,993 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:49,993 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:50,004 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:50,022 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:50,023 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:33:50,023 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:50,036 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:50,045 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:33:50,056 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:33:50,069 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.094705'} 2025-08-03 00:33:50,077 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:50,085 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.114942'} 2025-08-03 00:33:50,093 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:50,101 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:33:47', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.124978'} 2025-08-03 00:33:50,110 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:33:50,110 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:33:50,110 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:33:50,110 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:50,117 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:50,131 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:50,230 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:33:50,254 - DEBUG - ==>Verifying data for policy with id: 7cb38503-bbde-4086-8a06-1f6fb55b1275, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:33:50,254 - INFO - VN ctest-vn2-90001139 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:33:50,254 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:50,254 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:50,254 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:50,271 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:50,271 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:50,272 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:50,286 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:50,286 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:50,286 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:50,309 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:33:50,309 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:33:50,323 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2', '10.0.0.49': '3'} 2025-08-03 00:33:50,323 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:50,332 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:50,347 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:50,445 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:50,459 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:50,469 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:50,479 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:50,488 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:33:50,497 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:50,497 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:50,514 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:50,533 - INFO - Verified VN network id 14 for VN 57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:50,533 - INFO - Verifications in API Server for VN ctest-fip_vn-71509828 passed 2025-08-03 00:33:50,534 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:50,547 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:50,556 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:33:50,564 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:33:50,581 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:45.320076'} 2025-08-03 00:33:50,589 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:50,596 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:45.336148'} 2025-08-03 00:33:50,604 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:50,612 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:45.348879'} 2025-08-03 00:33:50,621 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:33:50,622 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-71509828 passed 2025-08-03 00:33:50,622 - DEBUG - ====Verifying policy data for ctest-fip_vn-71509828 in API_Server ====== 2025-08-03 00:33:50,622 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:33:50,630 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:33:50,645 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:33:50,745 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:33:50,762 - DEBUG - =>VN ctest-fip_vn-71509828 has no policy to be verified 2025-08-03 00:33:50,763 - DEBUG - Verifying the vn in opserver 2025-08-03 00:33:50,763 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.38 2025-08-03 00:33:50,763 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:33:50,777 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:33:50,777 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.254 2025-08-03 00:33:50,777 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:33:50,794 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:33:50,794 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.50 2025-08-03 00:33:50,794 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:33:50,810 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:33:50,810 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is found in opserver 2025-08-03 00:33:50,827 - DEBUG - VRF ids for VN ctest-fip_vn-71509828: {'10.0.0.48': '3'} 2025-08-03 00:33:50,827 - INFO - start time= 1754181230827774 2025-08-03 00:33:50,827 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:33:54,227 - DEBUG - file copy failed retrying 2025-08-03 00:33:57,616 - DEBUG - file copy failed retrying 2025-08-03 00:34:00,988 - DEBUG - file copy failed retrying 2025-08-03 00:34:04,444 - DEBUG - file copy failed retrying 2025-08-03 00:34:04,444 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:04,444 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoserver.py to 53.96.17.131 2025-08-03 00:34:04,444 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-55729628.pid --stats_file /tmp/server-ctest-random-55729628.stats 0<&- &> /tmp/server-ctest-random-55729628.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:34:04,444 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-55729628.pid --stats_file /tmp/server-ctest-random-55729628.stats 0<&- &> /tmp/server-ctest-random-55729628.log & 2025-08-03 00:34:04,964 - DEBUG - None 2025-08-03 00:34:08,623 - DEBUG - file copy failed retrying 2025-08-03 00:34:12,015 - DEBUG - file copy failed retrying 2025-08-03 00:34:15,419 - DEBUG - file copy failed retrying 2025-08-03 00:34:18,827 - DEBUG - file copy failed retrying 2025-08-03 00:34:18,827 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:18,827 - DEBUG - Copied file /contrail-test/tcutils/traffic_utils/../udpechoclient.py to 50.140.35.67 2025-08-03 00:34:18,827 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-55729628.pid --stats_file /tmp/client-ctest-random-55729628.stats 0<&- &> /tmp/client-ctest-random-55729628.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:34:18,827 - DEBUG - nohup python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-55729628.pid --stats_file /tmp/client-ctest-random-55729628.stats 0<&- &> /tmp/client-ctest-random-55729628.log & 2025-08-03 00:34:19,362 - DEBUG - None 2025-08-03 00:34:19,362 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:19,362 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:19,362 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 53.96.17.131 2025-08-03 00:34:19,362 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-47483009.pid --stats_file /tmp/server-ctest-random-47483009.stats 0<&- &> /tmp/server-ctest-random-47483009.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:34:19,362 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-47483009.pid --stats_file /tmp/server-ctest-random-47483009.stats 0<&- &> /tmp/server-ctest-random-47483009.log & 2025-08-03 00:34:19,857 - DEBUG - None 2025-08-03 00:34:19,857 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:19,857 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 50.140.35.67 2025-08-03 00:34:19,858 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-47483009.pid --stats_file /tmp/client-ctest-random-47483009.stats 0<&- &> /tmp/client-ctest-random-47483009.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:34:19,858 - DEBUG - nohup python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-47483009.pid --stats_file /tmp/client-ctest-random-47483009.stats 0<&- &> /tmp/client-ctest-random-47483009.log & 2025-08-03 00:34:20,079 - DEBUG - None 2025-08-03 00:34:20,079 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:20,079 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:20,079 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 53.96.17.131 2025-08-03 00:34:20,080 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-94542492.pid --stats_file /tmp/server-ctest-random-94542492.stats 0<&- &> /tmp/server-ctest-random-94542492.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:34:20,080 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-94542492.pid --stats_file /tmp/server-ctest-random-94542492.stats 0<&- &> /tmp/server-ctest-random-94542492.log & 2025-08-03 00:34:20,298 - DEBUG - None 2025-08-03 00:34:20,298 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:34:20,298 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 50.140.35.67 2025-08-03 00:34:20,298 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-94542492.pid --stats_file /tmp/client-ctest-random-94542492.stats 0<&- &> /tmp/client-ctest-random-94542492.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:34:20,298 - DEBUG - nohup python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-94542492.pid --stats_file /tmp/client-ctest-random-94542492.stats 0<&- &> /tmp/client-ctest-random-94542492.log & 2025-08-03 00:34:20,526 - DEBUG - None 2025-08-03 00:34:30,527 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-55729628.pid); sync; cat /tmp/client-ctest-random-55729628.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:34:30,528 - DEBUG - kill $(cat /tmp/client-ctest-random-55729628.pid); sync; cat /tmp/client-ctest-random-55729628.stats 2025-08-03 00:34:30,858 - DEBUG - cat: can't open '/tmp/client-ctest-random-55729628.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/client-ctest-random-55729628.stats': No such file or directory 2025-08-03 00:34:30,859 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-55729628.pid); sync; cat /tmp/server-ctest-random-55729628.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:34:30,859 - DEBUG - kill $(cat /tmp/server-ctest-random-55729628.pid); sync; cat /tmp/server-ctest-random-55729628.stats 2025-08-03 00:34:31,179 - DEBUG - cat: can't open '/tmp/server-ctest-random-55729628.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/server-ctest-random-55729628.stats': No such file or directory 2025-08-03 00:34:31,179 - INFO - Client - Sent: 0, Received: 0 2025-08-03 00:34:31,179 - INFO - Server - Sent: 0, Received: 0 2025-08-03 00:34:31,728 - INFO - Deleted policy policy1 2025-08-03 00:34:31,905 - INFO - Setting flow export rate: None 2025-08-03 00:34:34,894 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Sun Aug 3 00:34:31 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/analytics/test_analytics_resource.py in test_verify_session_record_table_inter_vn(self=) 1373 start_time = self.analytics_obj.getstarttime(vm_node_ip) 1374 self.logger.info("start time= %s" % (start_time)) 1375 self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) 1376 time.sleep(100) 1377 src_vn = self.res.vn1_fixture.vn_fq_name self = self.setup_and_create_streams = > self.res = self.res.vn1_vm1_fixture = self.res.vn2_vm2_fixture = /contrail-test/serial_scripts/analytics/base.py in setup_and_create_streams(self=, src_vm=, dst_vm=, sport=8000, dport=9003, count=100) 168 time.sleep(10) 169 for traffic_obj in traffic_objs: 170 self.stop_traffic(traffic_obj) 171 172 #end setup_create_streams self = self.stop_traffic = > traffic_obj = /contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False) 796 assert False, "No drops observed in packets %s"%msg 797 elif expectation: 798 assert sent and recv, "Packets not %s"%msg 799 if recv*100/float(sent) < 90: 800 assert False, "Packets not %s"%msg sent = 0 recv = 0 msg = 'transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000' AssertionError: Packets not transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000 __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('Packets not transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/analytics/test_analytics_resource.py", line 1375, in test_verify_session_record_table_inter_vn self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) File "/contrail-test/serial_scripts/analytics/base.py", line 170, in setup_and_create_streams self.stop_traffic(traffic_obj) File "/contrail-test/common/base.py", line 798, in stop_traffic assert sent and recv, "Packets not %s"%msg AssertionError: Packets not transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000 2025-08-03 00:34:34,894 - DEBUG - Skipping xmpp flap check 2025-08-03 00:34:34,895 - INFO -
2025-08-03 00:34:34,895 - INFO - END TEST : test_verify_session_record_table_inter_vn : FAILED[0:00:49]
2025-08-03 00:34:34,895 - INFO - -------------------------------------------------------------------------------- 2025-08-03 00:34:34,898 - INFO - ================================================================================ 2025-08-03 00:34:34,899 - INFO - STARTING TEST : test_verify_session_sampling_teardown 2025-08-03 00:34:34,899 - INFO - TEST DESCRIPTION : 1.query client session samples 2.query server session samples 3.query client session to get number of sessions exported 4.query session record table for teardown bytes/pkts 5.query sample count after teardown on server side 2025-08-03 00:34:36,289 - DEBUG - Skipping xmpp flap check 2025-08-03 00:34:36,289 - INFO - Initial checks done. Running the testcase now 2025-08-03 00:34:36,290 - INFO - 2025-08-03 00:34:36,545 - INFO - Setting flow export rate: 100 2025-08-03 00:34:36,597 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-08-03 00:34:37,066 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '175375e7-c1ac-461b-9825-88fae2d5f99d', 'name': 'ctest-vn1-24950719', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['b4152e6b-0e81-466c-85fe-9face7e7259f'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn1-24950719'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '50.140.35.64/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:34:37,342 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'aa59a538-60b9-42f1-b128-ae205a1a5f7d', 'name': 'ctest-vn2-90001139', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['ddafc3f1-5753-4d48-929e-fbcde432ab72'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn2-90001139'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '53.96.17.128/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:34:37,478 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:37,510 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:37,532 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:37,668 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:37,684 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:37,700 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:37,712 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:37,725 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:34:37,737 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:37,737 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:37,753 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:37,786 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:37,790 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:34:37,790 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:37,809 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:37,826 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:37,849 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:34:37,876 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:34:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.885460'} 2025-08-03 00:34:37,884 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:37,898 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:34:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.893732'} 2025-08-03 00:34:37,908 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:37,923 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:34:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.931389'} 2025-08-03 00:34:37,935 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:37,936 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:34:37,936 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:34:37,936 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:37,945 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:37,964 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:38,088 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:38,101 - DEBUG - ==>Verifying data for policy with id: ca090c9d-60a1-4e33-89f7-b2cb75ed9992, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:34:38,101 - INFO - VN ctest-vn1-24950719 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:34:38,101 - DEBUG - Verifying the vn in opserver 2025-08-03 00:34:38,101 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:34:38,101 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:34:38,119 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:34:38,119 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:34:38,119 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:34:38,146 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:34:38,146 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:34:38,146 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:34:38,165 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:34:38,165 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:34:38,179 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:34:38,179 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:38,188 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:38,206 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:38,324 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:38,348 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:38,365 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:38,378 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:38,393 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:34:38,402 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:38,402 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:38,415 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:38,442 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:38,442 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:34:38,443 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:38,465 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:38,477 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:38,488 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:34:38,507 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.219328'} 2025-08-03 00:34:38,516 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:38,526 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.232498'} 2025-08-03 00:34:38,542 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:38,551 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.266198'} 2025-08-03 00:34:38,565 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:38,565 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:34:38,565 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:34:38,565 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:38,572 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:38,586 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:38,694 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:38,708 - DEBUG - ==>Verifying data for policy with id: ca090c9d-60a1-4e33-89f7-b2cb75ed9992, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:34:38,708 - INFO - VN ctest-vn2-90001139 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:34:38,708 - DEBUG - Verifying the vn in opserver 2025-08-03 00:34:38,708 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:34:38,708 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:34:38,724 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:34:38,724 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:34:38,724 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:34:38,751 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:34:38,751 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:34:38,751 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:34:38,768 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:34:38,768 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:34:38,790 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2', '10.0.0.49': '3'} 2025-08-03 00:34:38,790 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:38,798 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:38,812 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:38,922 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:38,937 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:38,949 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:38,960 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:38,969 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:34:38,978 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:38,978 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:38,991 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:39,020 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:39,020 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:34:39,020 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:39,033 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:39,044 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:34:39,053 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:34:39,068 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:34:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.079160'} 2025-08-03 00:34:39,075 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:39,088 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:34:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.087202'} 2025-08-03 00:34:39,102 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:39,110 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:34:36', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.120188'} 2025-08-03 00:34:39,122 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:34:39,122 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:34:39,122 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:34:39,122 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:39,130 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:39,145 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:39,257 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:34:39,271 - DEBUG - ==>Verifying data for policy with id: ca090c9d-60a1-4e33-89f7-b2cb75ed9992, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:34:39,271 - INFO - VN ctest-vn1-24950719 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:34:39,271 - DEBUG - Verifying the vn in opserver 2025-08-03 00:34:39,271 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:34:39,271 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:34:39,286 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:34:39,286 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:34:39,286 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:34:39,311 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:34:39,311 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:34:39,311 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:34:39,329 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:34:39,329 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:34:39,342 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:34:39,343 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:39,350 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:39,363 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:39,477 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:39,489 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:39,502 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:39,513 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:39,523 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:34:39,533 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:39,533 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:39,546 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:39,578 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:39,578 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:34:39,578 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:39,592 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:39,603 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:34:39,614 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:34:39,630 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.343334'} 2025-08-03 00:34:39,638 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:39,651 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.357310'} 2025-08-03 00:34:39,664 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:39,680 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.394300'} 2025-08-03 00:34:39,691 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:34:39,692 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:34:39,692 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:34:39,692 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:39,699 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:39,714 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:39,826 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:34:39,842 - DEBUG - ==>Verifying data for policy with id: ca090c9d-60a1-4e33-89f7-b2cb75ed9992, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:34:39,842 - INFO - VN ctest-vn2-90001139 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:34:39,843 - DEBUG - Verifying the vn in opserver 2025-08-03 00:34:39,843 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:34:39,843 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:34:39,860 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:34:39,860 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:34:39,860 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:34:39,893 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:34:39,893 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:34:39,893 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:34:39,910 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:34:39,910 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:34:39,926 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2', '10.0.0.49': '3'} 2025-08-03 00:34:39,926 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:39,934 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:39,948 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:40,059 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:34:40,073 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:34:40,085 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:34:40,095 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:34:40,104 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:34:40,113 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:34:40,113 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:34:40,124 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:34:40,144 - INFO - Verified VN network id 14 for VN 57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:34:40,144 - INFO - Verifications in API Server for VN ctest-fip_vn-71509828 passed 2025-08-03 00:34:40,144 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:34:40,163 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:34:40,173 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:34:40,182 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:34:40,197 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:34.937456'} 2025-08-03 00:34:40,205 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:34:40,219 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:34.957509'} 2025-08-03 00:34:40,234 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:34:40,246 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:34.984082'} 2025-08-03 00:34:40,255 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:34:40,255 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-71509828 passed 2025-08-03 00:34:40,255 - DEBUG - ====Verifying policy data for ctest-fip_vn-71509828 in API_Server ====== 2025-08-03 00:34:40,256 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:34:40,263 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:34:40,278 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:34:40,405 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:34:40,421 - DEBUG - =>VN ctest-fip_vn-71509828 has no policy to be verified 2025-08-03 00:34:40,421 - DEBUG - Verifying the vn in opserver 2025-08-03 00:34:40,421 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.38 2025-08-03 00:34:40,421 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:34:40,437 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:34:40,437 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.254 2025-08-03 00:34:40,437 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:34:40,461 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:34:40,461 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.50 2025-08-03 00:34:40,461 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:34:40,484 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:34:40,484 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is found in opserver 2025-08-03 00:34:40,500 - DEBUG - VRF ids for VN ctest-fip_vn-71509828: {'10.0.0.48': '3'} 2025-08-03 00:34:40,501 - INFO - start time= 1754181280501048 2025-08-03 00:34:40,501 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 53.96.17.131, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:34:40,501 - DEBUG - ping -s 56 -c 3 -W 1 53.96.17.131 2025-08-03 00:34:42,745 - DEBUG - PING 53.96.17.131 (53.96.17.131): 56 data bytes 64 bytes from 53.96.17.131: seq=0 ttl=63 time=4.291 ms 64 bytes from 53.96.17.131: seq=1 ttl=63 time=1.503 ms 64 bytes from 53.96.17.131: seq=2 ttl=63 time=1.776 ms --- 53.96.17.131 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 1.503/2.523/4.291 ms 2025-08-03 00:34:42,746 - INFO - Ping to IP 53.96.17.131 from VM ctest-vn1_vm1-71948368 passed 2025-08-03 00:34:52,746 - INFO - Verify session samples and teardown pkts 2025-08-03 00:34:52,746 - INFO - Verifying SessionSeriesTable through opserver 10.0.0.38 2025-08-03 00:34:52,751 - DEBUG - Posting: http://10.0.0.38:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754181280501048, 'end_time': 1754181292751678, 'select_fields': ['T'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-08-03 00:34:53,777 - DEBUG - [{'T': 1754181281650835}, {'T': 1754181282650960}, {'T': 1754181283651027}] 2025-08-03 00:34:53,777 - DEBUG - Posting: http://10.0.0.38:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754181280501048, 'end_time': 1754181293777809, 'select_fields': ['T'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'server'} 2025-08-03 00:34:54,799 - DEBUG - [{'T': 1754181281522086}, {'T': 1754181282522196}, {'T': 1754181283522275}] 2025-08-03 00:34:54,799 - DEBUG - Posting: http://10.0.0.38:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754181280501048, 'end_time': 1754181294799188, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-08-03 00:34:55,833 - DEBUG - [{'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'sample_count': 3}] 2025-08-03 00:34:55,833 - INFO - wait for the flows to get expire 2025-08-03 00:38:15,834 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-3743-2:Compute:contrail-vrouter-agent:0?flat 2025-08-03 00:38:15,936 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3743-2?flat 2025-08-03 00:38:16,028 - DEBUG - Posting: http://10.0.0.38:8081/analytics/query, payload {'table': 'SessionRecordTable', 'start_time': 1754181280501048, 'end_time': 1754181496028541, 'select_fields': ['vn', 'remote_vn', 'forward_teardown_pkts', 'reverse_teardown_pkts'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-08-03 00:38:17,123 - DEBUG - [{'forward_flow_uuid': 'b646d441-e14d-462c-834a-f6bf44e763b7', 'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'reverse_flow_uuid': 'e32b7e83-c2c1-44b6-8693-867096cd5eba', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'forward_teardown_pkts': 3, 'reverse_teardown_pkts': 3}] 2025-08-03 00:38:17,123 - DEBUG - Posting: http://10.0.0.38:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754181280501048, 'end_time': 1754181497123526, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'} 2025-08-03 00:38:18,159 - DEBUG - Posting: http://10.0.0.38:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754181280501048, 'end_time': 1754181498159415, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'server'} 2025-08-03 00:38:19,185 - DEBUG - [{'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'sample_count': 4}] 2025-08-03 00:38:19,758 - INFO - Deleted policy policy1 2025-08-03 00:38:19,937 - INFO - Setting flow export rate: None 2025-08-03 00:38:21,240 - DEBUG - Skipping xmpp flap check
2025-08-03 00:38:21,240 - INFO - END TEST : test_verify_session_sampling_teardown : PASSED[0:03:47]
2025-08-03 00:38:21,240 - INFO - -------------------------------------------------------------------------------- 2025-08-03 00:38:21,247 - INFO - ================================================================================ 2025-08-03 00:38:21,247 - INFO - STARTING TEST : test_verify_session_series_table_inter_vn 2025-08-03 00:38:21,247 - INFO - TEST DESCRIPTION : Verify session series table ,generated stats between different vns 1.query client session samples 2.query for server ports 3.sort results by server_port column 4.verify granularity with T=10 5.verify sampled bytes 6.verify logged bytes 7.query and verify filter by action 2025-08-03 00:38:22,573 - DEBUG - Skipping xmpp flap check 2025-08-03 00:38:22,573 - INFO - Initial checks done. Running the testcase now 2025-08-03 00:38:22,573 - INFO - 2025-08-03 00:38:22,837 - INFO - Setting flow export rate: 100 2025-08-03 00:38:22,885 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = udp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2025-08-03 00:38:23,359 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '175375e7-c1ac-461b-9825-88fae2d5f99d', 'name': 'ctest-vn1-24950719', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['b4152e6b-0e81-466c-85fe-9face7e7259f'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn1-24950719'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '50.140.35.64/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:38:23,639 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'aa59a538-60b9-42f1-b128-ae205a1a5f7d', 'name': 'ctest-vn2-90001139', 'tenant_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'project_id': 'b4c83e3e011e46b38dfe709b7a6ac7d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['ddafc3f1-5753-4d48-929e-fbcde432ab72'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'ctest-vn2-90001139'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '53.96.17.128/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}} 2025-08-03 00:38:23,780 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:23,822 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:23,867 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:23,992 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:24,006 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:24,019 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:24,030 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:24,042 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:38:24,051 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:24,051 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:24,063 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:24,085 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:24,086 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:38:24,086 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:24,099 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:24,109 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:24,120 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:38:24,135 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.832305'} 2025-08-03 00:38:24,143 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:24,157 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.845969'} 2025-08-03 00:38:24,166 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:24,173 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.888889'} 2025-08-03 00:38:24,193 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:24,193 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:38:24,193 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:38:24,193 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:24,200 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:24,215 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:24,323 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:24,338 - DEBUG - ==>Verifying data for policy with id: 237d7dcd-271b-4a44-bc18-f601cf822759, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:38:24,338 - INFO - VN ctest-vn1-24950719 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:38:24,338 - DEBUG - Verifying the vn in opserver 2025-08-03 00:38:24,338 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:38:24,338 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:38:24,354 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:38:24,354 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:38:24,354 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:38:24,432 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:38:24,432 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:38:24,432 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:38:24,512 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:38:24,512 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:38:24,528 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:38:24,528 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:24,538 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:24,555 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:24,667 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:24,691 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:24,703 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:24,713 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:24,724 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:38:24,734 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:24,734 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:24,745 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:24,768 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:24,768 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:38:24,768 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:24,780 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:24,789 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:24,798 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:38:24,811 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.234654'} 2025-08-03 00:38:24,818 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:24,831 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.247698'} 2025-08-03 00:38:24,844 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:24,858 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.278708'} 2025-08-03 00:38:24,875 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:24,875 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:38:24,875 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:38:24,875 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:24,883 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:24,896 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:25,020 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:25,034 - DEBUG - ==>Verifying data for policy with id: 237d7dcd-271b-4a44-bc18-f601cf822759, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:38:25,034 - INFO - VN ctest-vn2-90001139 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:38:25,034 - DEBUG - Verifying the vn in opserver 2025-08-03 00:38:25,034 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:38:25,034 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:38:25,052 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:38:25,052 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:38:25,052 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:38:25,140 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:38:25,141 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:38:25,141 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:38:25,252 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:38:25,252 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:38:25,266 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2', '10.0.0.49': '3'} 2025-08-03 00:38:25,266 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:25,274 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:25,287 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:25,408 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:25,422 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:25,434 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:25,445 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:25,455 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:38:25,464 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:25,464 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:25,478 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:25,511 - INFO - Verified VN network id 12 for VN 175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:25,511 - INFO - Verifications in API Server for VN ctest-vn1-24950719 passed 2025-08-03 00:38:25,511 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:25,524 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:25,534 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:25,544 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/00fa9e41-39bb-435a-815b-ccdf576d1030 2025-08-03 00:38:25,557 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.255002'} 2025-08-03 00:38:25,564 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:25,577 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.265619'} 2025-08-03 00:38:25,594 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:25,607 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1680816724186514971', 'uuid-lslong': '10963319479032215965'}, 'enable': 'true', 'created': '2025-08-03T00:32:26', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.323217'} 2025-08-03 00:38:25,623 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-08-03 00:38:25,623 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-24950719 passed 2025-08-03 00:38:25,623 - DEBUG - ====Verifying policy data for ctest-vn1-24950719 in API_Server ====== 2025-08-03 00:38:25,623 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:25,631 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:25,645 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:25,763 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/175375e7-c1ac-461b-9825-88fae2d5f99d 2025-08-03 00:38:25,779 - DEBUG - ==>Verifying data for policy with id: 237d7dcd-271b-4a44-bc18-f601cf822759, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:38:25,779 - INFO - VN ctest-vn1-24950719 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:38:25,779 - DEBUG - Verifying the vn in opserver 2025-08-03 00:38:25,779 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.38 2025-08-03 00:38:25,779 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:38:25,801 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:38:25,801 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.254 2025-08-03 00:38:25,801 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:38:25,830 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:38:25,830 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 virtual network link through opserver 10.0.0.50 2025-08-03 00:38:25,831 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:38:25,852 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719?flat'} 2025-08-03 00:38:25,853 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 is found in opserver 2025-08-03 00:38:25,866 - DEBUG - VRF ids for VN ctest-vn1-24950719: {'10.0.0.49': '2'} 2025-08-03 00:38:25,866 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:25,874 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:25,888 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:26,030 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:26,044 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:26,061 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:26,073 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:26,083 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:38:26,092 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:26,092 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:26,104 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:26,135 - INFO - Verified VN network id 13 for VN aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:26,135 - INFO - Verifications in API Server for VN ctest-vn2-90001139 passed 2025-08-03 00:38:26,135 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:26,147 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:26,157 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:26,167 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/79e1ff5e-4285-4815-b28d-50f456fb2f7c 2025-08-03 00:38:26,185 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '0-1', 'advertised': '0-1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.608020'} 2025-08-03 00:38:26,194 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:26,209 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.623257'} 2025-08-03 00:38:26,226 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:26,242 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12275023920934503153', 'uuid-lslong': '12765644597755535229'}, 'enable': 'true', 'created': '2025-08-03T00:33:04', 'last-modified': '2025-08-03T00:38:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.661899'} 2025-08-03 00:38:26,256 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-08-03 00:38:26,256 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-90001139 passed 2025-08-03 00:38:26,256 - DEBUG - ====Verifying policy data for ctest-vn2-90001139 in API_Server ====== 2025-08-03 00:38:26,256 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:26,266 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:26,284 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:26,405 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/aa59a538-60b9-42f1-b128-ae205a1a5f7d 2025-08-03 00:38:26,420 - DEBUG - ==>Verifying data for policy with id: 237d7dcd-271b-4a44-bc18-f601cf822759, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-81719282', 'policy1'] 2025-08-03 00:38:26,421 - INFO - VN ctest-vn2-90001139 Policy verification: verify_vn_policy_in_api_server, status: True 2025-08-03 00:38:26,421 - DEBUG - Verifying the vn in opserver 2025-08-03 00:38:26,421 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.38 2025-08-03 00:38:26,421 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:38:26,443 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:38:26,443 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.254 2025-08-03 00:38:26,443 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:38:26,474 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:38:26,474 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 virtual network link through opserver 10.0.0.50 2025-08-03 00:38:26,474 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:38:26,505 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139?flat'} 2025-08-03 00:38:26,505 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 is found in opserver 2025-08-03 00:38:26,520 - DEBUG - VRF ids for VN ctest-vn2-90001139: {'10.0.0.48': '2', '10.0.0.49': '3'} 2025-08-03 00:38:26,520 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:26,529 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:26,545 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:26,662 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:38:26,678 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:38:26,690 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:38:26,702 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:38:26,713 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:38:26,723 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:38:26,723 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:38:26,736 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:38:26,766 - INFO - Verified VN network id 14 for VN 57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:38:26,767 - INFO - Verifications in API Server for VN ctest-fip_vn-71509828 passed 2025-08-03 00:38:26,768 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:38:26,781 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:38:26,791 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:38:26,803 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/80f79573-8518-4f26-bb74-cd275811aba4 2025-08-03 00:38:26,821 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '1', 'advertised': '1', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:21.560893'} 2025-08-03 00:38:26,828 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:38:26,842 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:21.581365'} 2025-08-03 00:38:26,857 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:38:26,870 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-81719282', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6338600124581561112', 'uuid-lslong': '13186602400913939017'}, 'enable': 'true', 'created': '2025-08-03T00:33:05', 'last-modified': '2025-08-03T00:33:05', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:21.607707'} 2025-08-03 00:38:26,880 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-08-03 00:38:26,880 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-71509828 passed 2025-08-03 00:38:26,880 - DEBUG - ====Verifying policy data for ctest-fip_vn-71509828 in API_Server ====== 2025-08-03 00:38:26,880 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:26,890 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:26,907 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:27,040 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/57f73b3a-deee-4318-b700-39049cc9da49 2025-08-03 00:38:27,057 - DEBUG - =>VN ctest-fip_vn-71509828 has no policy to be verified 2025-08-03 00:38:27,057 - DEBUG - Verifying the vn in opserver 2025-08-03 00:38:27,057 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.38 2025-08-03 00:38:27,057 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-08-03 00:38:27,075 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:38:27,075 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.254 2025-08-03 00:38:27,075 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-08-03 00:38:27,147 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:38:27,147 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 virtual network link through opserver 10.0.0.50 2025-08-03 00:38:27,147 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-08-03 00:38:27,244 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828?flat'} 2025-08-03 00:38:27,244 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 is found in opserver 2025-08-03 00:38:27,258 - DEBUG - VRF ids for VN ctest-fip_vn-71509828: {'10.0.0.48': '3'} 2025-08-03 00:38:27,258 - INFO - start time= 1754181507258725 2025-08-03 00:38:27,258 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:27,259 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:27,259 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 53.96.17.131 2025-08-03 00:38:27,259 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-60121760.pid --stats_file /tmp/server-ctest-random-60121760.stats 0<&- &> /tmp/server-ctest-random-60121760.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:38:27,259 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-60121760.pid --stats_file /tmp/server-ctest-random-60121760.stats 0<&- &> /tmp/server-ctest-random-60121760.log & 2025-08-03 00:38:27,474 - DEBUG - None 2025-08-03 00:38:27,474 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:27,474 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 50.140.35.67 2025-08-03 00:38:27,475 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-60121760.pid --stats_file /tmp/client-ctest-random-60121760.stats 0<&- &> /tmp/client-ctest-random-60121760.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:38:27,475 - DEBUG - nohup python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-60121760.pid --stats_file /tmp/client-ctest-random-60121760.stats 0<&- &> /tmp/client-ctest-random-60121760.log & 2025-08-03 00:38:27,696 - DEBUG - None 2025-08-03 00:38:27,696 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:27,696 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:27,696 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 53.96.17.131 2025-08-03 00:38:27,696 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-84877640.pid --stats_file /tmp/server-ctest-random-84877640.stats 0<&- &> /tmp/server-ctest-random-84877640.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:38:27,696 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-84877640.pid --stats_file /tmp/server-ctest-random-84877640.stats 0<&- &> /tmp/server-ctest-random-84877640.log & 2025-08-03 00:38:27,912 - DEBUG - None 2025-08-03 00:38:27,912 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:27,912 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 50.140.35.67 2025-08-03 00:38:27,912 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-84877640.pid --stats_file /tmp/client-ctest-random-84877640.stats 0<&- &> /tmp/client-ctest-random-84877640.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:38:27,912 - DEBUG - nohup python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-84877640.pid --stats_file /tmp/client-ctest-random-84877640.stats 0<&- &> /tmp/client-ctest-random-84877640.log & 2025-08-03 00:38:28,160 - DEBUG - None 2025-08-03 00:38:28,161 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:28,161 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:28,161 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 53.96.17.131 2025-08-03 00:38:28,161 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-61117470.pid --stats_file /tmp/server-ctest-random-61117470.stats 0<&- &> /tmp/server-ctest-random-61117470.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:38:28,161 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-61117470.pid --stats_file /tmp/server-ctest-random-61117470.stats 0<&- &> /tmp/server-ctest-random-61117470.log & 2025-08-03 00:38:28,380 - DEBUG - None 2025-08-03 00:38:28,380 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-08-03 00:38:28,380 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 50.140.35.67 2025-08-03 00:38:28,380 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-61117470.pid --stats_file /tmp/client-ctest-random-61117470.stats 0<&- &> /tmp/client-ctest-random-61117470.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:38:28,380 - DEBUG - nohup python /tmp/udpechoclient.py --servers 53.96.17.131 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-61117470.pid --stats_file /tmp/client-ctest-random-61117470.stats 0<&- &> /tmp/client-ctest-random-61117470.log & 2025-08-03 00:38:28,596 - DEBUG - None 2025-08-03 00:38:38,597 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-60121760.pid); sync; cat /tmp/client-ctest-random-60121760.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.49, gateway password: c0ntrail123 2025-08-03 00:38:38,597 - DEBUG - kill $(cat /tmp/client-ctest-random-60121760.pid); sync; cat /tmp/client-ctest-random-60121760.stats 2025-08-03 00:38:38,910 - DEBUG - cat: can't open '/tmp/client-ctest-random-60121760.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/client-ctest-random-60121760.stats': No such file or directory 2025-08-03 00:38:38,910 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-60121760.pid); sync; cat /tmp/server-ctest-random-60121760.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.48, gateway password: c0ntrail123 2025-08-03 00:38:38,910 - DEBUG - kill $(cat /tmp/server-ctest-random-60121760.pid); sync; cat /tmp/server-ctest-random-60121760.stats 2025-08-03 00:38:39,234 - DEBUG - cat: can't open '/tmp/server-ctest-random-60121760.pid': No such file or directory sh: you need to specify whom to kill cat: can't open '/tmp/server-ctest-random-60121760.stats': No such file or directory 2025-08-03 00:38:39,234 - INFO - Client - Sent: 0, Received: 0 2025-08-03 00:38:39,234 - INFO - Server - Sent: 0, Received: 0 2025-08-03 00:38:39,721 - INFO - Deleted policy policy1 2025-08-03 00:38:39,885 - INFO - Setting flow export rate: None 2025-08-03 00:38:41,269 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Sun Aug 3 00:38:39 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/analytics/test_analytics_resource.py in test_verify_session_series_table_inter_vn(self=) 1336 start_time = self.analytics_obj.getstarttime(vm_node_ip) 1337 self.logger.info("start time= %s" % (start_time)) 1338 self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) 1339 src_vn = self.res.vn1_fixture.vn_fq_name 1340 dst_vn = self.res.vn2_fixture.vn_fq_name self = self.setup_and_create_streams = > self.res = self.res.vn1_vm1_fixture = self.res.vn2_vm2_fixture = /contrail-test/serial_scripts/analytics/base.py in setup_and_create_streams(self=, src_vm=, dst_vm=, sport=8000, dport=9003, count=100) 168 time.sleep(10) 169 for traffic_obj in traffic_objs: 170 self.stop_traffic(traffic_obj) 171 172 #end setup_create_streams self = self.stop_traffic = > traffic_obj = /contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False) 796 assert False, "No drops observed in packets %s"%msg 797 elif expectation: 798 assert sent and recv, "Packets not %s"%msg 799 if recv*100/float(sent) < 90: 800 assert False, "Packets not %s"%msg sent = 0 recv = 0 msg = 'transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000' AssertionError: Packets not transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000 __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ('Packets not transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000',) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/analytics/test_analytics_resource.py", line 1338, in test_verify_session_series_table_inter_vn self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture) File "/contrail-test/serial_scripts/analytics/base.py", line 170, in setup_and_create_streams self.stop_traffic(traffic_obj) File "/contrail-test/common/base.py", line 798, in stop_traffic assert sent and recv, "Packets not %s"%msg AssertionError: Packets not transferred between 50.140.35.67 and 53.96.17.131, proto udp sport 8000 and dport 9000 2025-08-03 00:38:41,270 - DEBUG - Skipping xmpp flap check 2025-08-03 00:38:41,270 - INFO -
2025-08-03 00:38:41,270 - INFO - END TEST : test_verify_session_series_table_inter_vn : FAILED[0:00:20]
2025-08-03 00:38:41,270 - INFO - -------------------------------------------------------------------------------- 2025-08-03 00:38:41,272 - INFO - Deleting VM mltf_vm 2025-08-03 00:38:41,367 - INFO - Deleting VM ctest-fvn_vm1-96269498 2025-08-03 00:38:41,455 - INFO - Deleting VM ctest-vn2_vm2-50961308 2025-08-03 00:38:41,551 - INFO - Deleting VN ctest-fip_vn-71509828 2025-08-03 00:38:41,633 - DEBUG - VN 57f73b3a-deee-4318-b700-39049cc9da49 still in use: Unable to complete operation on network 57f73b3a-deee-4318-b700-39049cc9da49. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-92e88944-b18a-4caa-8e18-6d9008522235'] 2025-08-03 00:38:41,633 - WARNING - Deleting VN ctest-fip_vn-71509828 failed..Will retry 2025-08-03 00:38:43,840 - DEBUG - Response for deleting network () 2025-08-03 00:38:43,841 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/0eabdc6f-1d1c-41ec-aef7-3124fef69c44 2025-08-03 00:38:43,853 - DEBUG - Response Code: 404 2025-08-03 00:38:43,854 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:43,866 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:43,886 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:44,028 - INFO - Validated that VN ctest-fip_vn-71509828 is not found in API Server 2025-08-03 00:38:44,042 - DEBUG - VN ctest-fip_vn-71509828 is not present in Agent 10.0.0.48 2025-08-03 00:38:44,055 - DEBUG - VN ctest-fip_vn-71509828 is not present in Agent 10.0.0.49 2025-08-03 00:38:44,055 - INFO - Validated that VN ctest-fip_vn-71509828 is not in any agent 2025-08-03 00:38:44,063 - DEBUG - VRF 3 is not seen in agent 10.0.0.48 2025-08-03 00:38:44,070 - DEBUG - Vrouter 10.0.0.48 does not have vrf 3 for VN ctest-fip_vn-71509828 2025-08-03 00:38:44,070 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-fip_vn-71509828 2025-08-03 00:38:44,102 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-fip_vn-71509828 info 2025-08-03 00:38:44,102 - INFO - Deleting VN ctest-vn2-90001139 2025-08-03 00:38:44,363 - DEBUG - Response for deleting network () 2025-08-03 00:38:44,363 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/f2190c96-a7b7-42d0-a59c-16cb79307933 2025-08-03 00:38:44,372 - DEBUG - Response Code: 404 2025-08-03 00:38:44,372 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:44,407 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:44,438 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:44,588 - INFO - Validated that VN ctest-vn2-90001139 is not found in API Server 2025-08-03 00:38:44,610 - DEBUG - VN ctest-vn2-90001139 is not present in Agent 10.0.0.48 2025-08-03 00:38:44,632 - DEBUG - VN ctest-vn2-90001139 is not present in Agent 10.0.0.49 2025-08-03 00:38:44,632 - INFO - Validated that VN ctest-vn2-90001139 is not in any agent 2025-08-03 00:38:44,639 - DEBUG - VRF 2 is not seen in agent 10.0.0.48 2025-08-03 00:38:44,650 - DEBUG - Vrouter 10.0.0.48 does not have vrf 2 for VN ctest-vn2-90001139 2025-08-03 00:38:44,660 - DEBUG - VRF 3 is not seen in agent 10.0.0.49 2025-08-03 00:38:44,667 - DEBUG - Vrouter 10.0.0.49 does not have vrf 3 for VN ctest-vn2-90001139 2025-08-03 00:38:44,667 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn2-90001139 2025-08-03 00:38:44,702 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn2-90001139 info 2025-08-03 00:38:44,703 - INFO - Deleting VM ctest-vn1_vm2-57667192 2025-08-03 00:38:44,790 - INFO - Deleting VM ctest-vn1_vm1-71948368 2025-08-03 00:38:44,926 - INFO - Deleting VN ctest-vn1-24950719 2025-08-03 00:38:44,987 - DEBUG - VN 175375e7-c1ac-461b-9825-88fae2d5f99d still in use: Unable to complete operation on network 175375e7-c1ac-461b-9825-88fae2d5f99d. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-7477162d-c657-48dd-abc0-9812475f20df'] 2025-08-03 00:38:44,987 - WARNING - Deleting VN ctest-vn1-24950719 failed..Will retry 2025-08-03 00:38:47,205 - DEBUG - Response for deleting network () 2025-08-03 00:38:47,205 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/5dec7036-4606-4715-8f4c-5d55a7e5733d 2025-08-03 00:38:47,214 - DEBUG - Response Code: 404 2025-08-03 00:38:47,214 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-08-03 00:38:47,220 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e48a4202-dd3d-4b87-9406-48f7b2154028 2025-08-03 00:38:47,281 - DEBUG - Requesting: http://10.0.0.38:8082/project/b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0 2025-08-03 00:38:47,406 - INFO - Validated that VN ctest-vn1-24950719 is not found in API Server 2025-08-03 00:38:47,419 - DEBUG - VN ctest-vn1-24950719 is not present in Agent 10.0.0.48 2025-08-03 00:38:47,432 - DEBUG - VN ctest-vn1-24950719 is not present in Agent 10.0.0.49 2025-08-03 00:38:47,432 - INFO - Validated that VN ctest-vn1-24950719 is not in any agent 2025-08-03 00:38:47,438 - DEBUG - VRF 2 is not seen in agent 10.0.0.49 2025-08-03 00:38:47,444 - DEBUG - Vrouter 10.0.0.49 does not have vrf 2 for VN ctest-vn1-24950719 2025-08-03 00:38:47,444 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-81719282:ctest-vn1-24950719 2025-08-03 00:38:47,478 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn1-24950719 info 2025-08-03 00:38:48,266 - INFO - Deleted project: ctest-AnalyticsTestSanityWithResource-81719282, ID : b4c83e3e-011e-46b3-8dfe-709b7a6ac7d0