2025-11-27 04:12:39,245 - INFO - Domain Default found not creating 2025-11-27 04:12:39,416 - INFO - Project ctest-TestDropStats-65491859 not found, creating it 2025-11-27 04:12:39,960 - INFO - Created Project:ctest-TestDropStats-65491859, ID : 367ce5ba-6bb9-4dd4-afb8-b1ca7effca72 2025-11-27 04:12:41,462 - INFO - ================================================================================ 2025-11-27 04:12:41,462 - INFO - STARTING TEST : test_flow_action_drop_stats 2025-11-27 04:12:41,462 - INFO - TEST DESCRIPTION : Description: Verify flow action drop stats Steps: 1. Create 2 VNs and launch 1 VM in each VN, on different nodes on multi node setup 2. Add policy to deny pkts between VNs. 3. Ping VM2 from VM1 Pass criteria: 1. Verify flow action count gets incremented, get it from introspect 2. Verify flow action count gets incremented for the vmi of VM1, get it from introspect 2025-11-27 04:12:42,748 - DEBUG - Skipping xmpp flap check 2025-11-27 04:12:42,748 - INFO - Initial checks done. Running the testcase now 2025-11-27 04:12:42,748 - INFO - 2025-11-27 04:12:42,779 - DEBUG - Services list from nova: [, , , ] 2025-11-27 04:12:42,779 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4820-2', 'an-jenkins-deploy-platform-ansible-os-4820-1']} 2025-11-27 04:12:43,391 - DEBUG - Response for create_network : {'network': {'id': '42fb88dc-f485-4680-9077-0f12ebee3450', 'name': 'ctest-vn-75402471', 'tenant_id': '367ce5ba6bb94dd4afb8b1ca7effca72', 'project_id': '367ce5ba6bb94dd4afb8b1ca7effca72', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-65491859', 'ctest-vn-75402471'], 'port_security_enabled': True, 'description': ''}} 2025-11-27 04:12:43,671 - DEBUG - Response for create_subnet : {'subnet': {'id': 'ce25124b-1904-4997-b135-c9811f8338a3', 'name': '', 'tenant_id': '367ce5ba6bb94dd4afb8b1ca7effca72', 'network_id': '42fb88dc-f485-4680-9077-0f12ebee3450', 'ip_version': 4, 'cidr': '133.19.126.64/26', 'allocation_pools': [{'start': '133.19.126.66', 'end': '133.19.126.126'}], 'gateway_ip': '133.19.126.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '133.19.126.66', 'tags': [], 'project_id': '367ce5ba6bb94dd4afb8b1ca7effca72'}} 2025-11-27 04:12:43,693 - INFO - Created VN ctest-vn-75402471 2025-11-27 04:12:43,705 - DEBUG - VN ctest-vn-75402471 UUID is 42fb88dc-f485-4680-9077-0f12ebee3450 2025-11-27 04:12:44,070 - DEBUG - Response for create_network : {'network': {'id': '92a36183-0312-42f4-b932-0fd99b5cc02f', 'name': 'ctest-vn-69183138', 'tenant_id': '367ce5ba6bb94dd4afb8b1ca7effca72', 'project_id': '367ce5ba6bb94dd4afb8b1ca7effca72', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-65491859', 'ctest-vn-69183138'], 'port_security_enabled': True, 'description': ''}} 2025-11-27 04:12:44,331 - DEBUG - Response for create_subnet : {'subnet': {'id': '74eb6f5b-15bd-4585-872e-986603b634c1', 'name': '', 'tenant_id': '367ce5ba6bb94dd4afb8b1ca7effca72', 'network_id': '92a36183-0312-42f4-b932-0fd99b5cc02f', 'ip_version': 4, 'cidr': '171.235.56.64/26', 'allocation_pools': [{'start': '171.235.56.66', 'end': '171.235.56.126'}], 'gateway_ip': '171.235.56.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '171.235.56.66', 'tags': [], 'project_id': '367ce5ba6bb94dd4afb8b1ca7effca72'}} 2025-11-27 04:12:44,352 - INFO - Created VN ctest-vn-69183138 2025-11-27 04:12:44,362 - DEBUG - VN ctest-vn-69183138 UUID is 92a36183-0312-42f4-b932-0fd99b5cc02f 2025-11-27 04:12:45,496 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4820-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4820-1) 2025-11-27 04:12:46,785 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4820-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4820-2) 2025-11-27 04:12:46,785 - INFO - Waiting for VM ctest-vm1-76104767 to be up.. 2025-11-27 04:12:46,873 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-27 04:12:51,957 - DEBUG - VM is in ACTIVE state now 2025-11-27 04:12:51,957 - INFO - VM name : ctest-vm1-76104767 2025-11-27 04:12:52,048 - DEBUG - VM ctest-vm1-76104767 ID is e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:12:52,072 - DEBUG - VM ctest-vm1-76104767 launched on Node an-jenkins-deploy-platform-ansible-os-4820-1 2025-11-27 04:12:52,146 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:12:52,447 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:12:52,487 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/3156545a-3851-4915-a806-30efc0fdb36e 2025-11-27 04:12:55,656 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-11-27 04:12:55,656 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vm1-76104767 failed! 2025-11-27 04:12:55,672 - DEBUG - Gateway for vn default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 is 133.19.126.65 and allocation pool is NOT set 2025-11-27 04:12:57,734 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=7.97 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.527 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.527/4.248/7.970/3.721 ms') 2025-11-27 04:12:57,734 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vm1-76104767 passed 2025-11-27 04:12:57,801 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-27 04:12:57,801 - DEBUG - Waiting to SSH to VM ctest-vm1-76104767, IP 133.19.126.67, Port 22 2025-11-27 04:12:57,955 - DEBUG - VM ctest-vm1-76104767 is ready for SSH connections 2025-11-27 04:12:57,955 - INFO - Waiting for VM ctest-vm2-38853728 to be up.. 2025-11-27 04:12:58,062 - DEBUG - VM is in ACTIVE state now 2025-11-27 04:12:58,063 - INFO - VM name : ctest-vm2-38853728 2025-11-27 04:12:58,146 - DEBUG - VM ctest-vm2-38853728 ID is 1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:12:58,147 - DEBUG - VM ctest-vm2-38853728 launched on Node an-jenkins-deploy-platform-ansible-os-4820-2 2025-11-27 04:12:58,235 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:12:58,244 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/87e01ad3-38b3-4628-992a-c95f3ccca740 2025-11-27 04:12:59,406 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.25 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.693 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.693/2.970/5.248/2.277 ms') 2025-11-27 04:12:59,406 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vm2-38853728 passed 2025-11-27 04:12:59,473 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-27 04:12:59,473 - DEBUG - Waiting to SSH to VM ctest-vm2-38853728, IP 171.235.56.67, Port 22 2025-11-27 04:12:59,639 - DEBUG - VM ctest-vm2-38853728 is ready for SSH connections 2025-11-27 04:12:59,716 - DEBUG - VM is in ACTIVE state now 2025-11-27 04:12:59,716 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-27 04:12:59,716 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:12:59,724 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/3156545a-3851-4915-a806-30efc0fdb36e 2025-11-27 04:12:59,733 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:12:59,740 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/3156545a-3851-4915-a806-30efc0fdb36e 2025-11-27 04:12:59,748 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/7cd57d6e-9982-490a-b98d-f7ff08f8da0c 2025-11-27 04:12:59,756 - DEBUG - Verifying in api server 10.0.0.23 2025-11-27 04:12:59,756 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:12:59,763 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/3156545a-3851-4915-a806-30efc0fdb36e 2025-11-27 04:12:59,772 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/7cd57d6e-9982-490a-b98d-f7ff08f8da0c 2025-11-27 04:12:59,778 - INFO - VM ctest-vm1-76104767 verfication in all API Servers passed 2025-11-27 04:12:59,801 - DEBUG - VM ctest-vm1-76104767 Tap interface: {'index': '3', 'name': 'tap3156545a-38', 'uuid': '3156545a-3851-4915-a806-30efc0fdb36e', 'vrf_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471:ctest-vn-75402471', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '25', 'l2_label': '29', 'vxlan_id': '6', 'vn_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471', 'vm_uuid': 'e5d1d560-83b6-4bad-a40b-4d1f414bca54', 'vm_name': 'ctest-vm1-76104767', 'ip_addr': '133.19.126.67', 'mac_addr': '02:31:56:54:5a:38', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '134', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e', 'sg_uuid_list': ['7b76b989-6e3d-4669-b9f8-b2c2b64d0094'], 'static_route_list': None, 'vm_project_uuid': '367ce5ba-6bb9-4dd4-afb8-b1ca7effca72', 'admin_state': 'Enabled', 'flow_key_idx': '26', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-11-27 04:12:59,801 - DEBUG - Agent 10.0.0.19 vrf name: default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471:ctest-vn-75402471 2025-11-27 04:12:59,808 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471:ctest-vn-75402471', 'ucindex': '2', 'mcindex': '2', 'l2index': '2', 'source': 'Config; ', 'uc6index': '2', 'vn': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471', 'table_label': '-1', 'vxlan_id': '6', 'evpnindex': '2', 'brindex': '2', 'mplsindex': '0', 'RD': '10.20.0.18:2', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2025-11-27 04:12:59,882 - DEBUG - VM ctest-vm1-76104767 : Tap interface tap3156545a-38 is set to Active 2025-11-27 04:12:59,882 - DEBUG - Tap interface tap3156545a-38 detail : {'index': '3', 'name': 'tap3156545a-38', 'uuid': '3156545a-3851-4915-a806-30efc0fdb36e', 'vrf_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471:ctest-vn-75402471', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '25', 'l2_label': '29', 'vxlan_id': '6', 'vn_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471', 'vm_uuid': 'e5d1d560-83b6-4bad-a40b-4d1f414bca54', 'vm_name': 'ctest-vm1-76104767', 'ip_addr': '133.19.126.67', 'mac_addr': '02:31:56:54:5a:38', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '134', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e', 'sg_uuid_list': ['7b76b989-6e3d-4669-b9f8-b2c2b64d0094'], 'static_route_list': None, 'vm_project_uuid': '367ce5ba-6bb9-4dd4-afb8-b1ca7effca72', 'admin_state': 'Enabled', 'flow_key_idx': '26', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-11-27 04:12:59,989 - DEBUG - Starting Layer 2 verification in Agent 2025-11-27 04:13:00,000 - DEBUG - Layer 2 path is seen for VM MAC 02:31:56:54:5a:38 in agent 10.0.0.19 2025-11-27 04:13:00,000 - DEBUG - Active layer 2 route in agent is present for VMI tap3156545a-38 2025-11-27 04:13:00,000 - DEBUG - L2 label(29) matches bw route table and interface table 2025-11-27 04:13:00,007 - DEBUG - VN default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 is found in Agent of node 10.0.0.19 2025-11-27 04:13:00,129 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 is consistent in agent 10.0.0.19 2025-11-27 04:13:00,129 - DEBUG - Route for VM IP ['133.19.126.67'] is consistent in agent 10.0.0.19 2025-11-27 04:13:00,129 - DEBUG - VN default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 verification for VM ctest-vm1-76104767 in Agent 10.0.0.19 passed 2025-11-27 04:13:00,240 - DEBUG - Starting all layer 2 verification in agent 10.0.0.19 2025-11-27 04:13:00,252 - DEBUG - Route for VM MAC 02:31:56:54:5a:38 is consistent in agent 10.0.0.19 2025-11-27 04:13:00,258 - INFO - VM ctest-vm1-76104767 verifications in Compute nodes passed 2025-11-27 04:13:00,494 - DEBUG - Validated VM route 133.19.126.67 in vrouter of 10.0.0.19 2025-11-27 04:13:00,499 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2025-11-27 04:13:00,499 - INFO - Validated routes of VM ctest-vm1-76104767 in all vrouters 2025-11-27 04:13:01,022 - DEBUG - Starting all layer2 verification in 10.0.0.27 Control Node 2025-11-27 04:13:01,150 - DEBUG - Layer2 route found for VM MAC 02:31:56:54:5a:38 in Control-node 10.0.0.27 2025-11-27 04:13:01,150 - DEBUG - L2 Label for VM ctest-vm1-76104767 same between Control-node 10.0.0.27 and Agent, Expected: 29, Seen: 29 2025-11-27 04:13:01,151 - DEBUG - Starting all layer2 verification in 10.0.0.33 Control Node 2025-11-27 04:13:01,281 - DEBUG - Layer2 route found for VM MAC 02:31:56:54:5a:38 in Control-node 10.0.0.33 2025-11-27 04:13:01,282 - DEBUG - L2 Label for VM ctest-vm1-76104767 same between Control-node 10.0.0.33 and Agent, Expected: 29, Seen: 29 2025-11-27 04:13:01,282 - INFO - Verification in Control-nodes for VM ctest-vm1-76104767 passed 2025-11-27 04:13:01,282 - DEBUG - Verifying the vm in opserver 2025-11-27 04:13:01,282 - DEBUG - Verifying in collector 10.0.0.23 ... 2025-11-27 04:13:01,282 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat 2025-11-27 04:13:01,585 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat 2025-11-27 04:13:01,652 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e?flat 2025-11-27 04:13:01,953 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e?flat 2025-11-27 04:13:02,023 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e?flat 2025-11-27 04:13:02,095 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat 2025-11-27 04:13:02,160 - DEBUG - Verifying vm in vn uve 2025-11-27 04:13:02,160 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e?flat 2025-11-27 04:13:02,231 - DEBUG - VM uve shows interface as {'port_mirror_enabled': False, 'ip_address': '133.19.126.67', 'ip6_address': '::', 'active': True, 'virtual_network': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471', 'ip6_active': False, 'rx_vlan': 65535, 'mac_address': '02:31:56:54:5a:38', 'vn_uuid': '42fb88dc-f485-4680-9077-0f12ebee3450', 'label': 25, 'uuid': '3156545a-3851-4915-a806-30efc0fdb36e', 'is_health_check_active': True, 'vm_uuid': 'e5d1d560-83b6-4bad-a40b-4d1f414bca54', 'vhostuser_mode': 0, 'tx_vlan': 65535, 'admin_state': True, 'vm_name': 'ctest-vm1-76104767', 'l2_active': True, 'gateway': '133.19.126.65', 'ip4_active': True, '__T': 1764216776243680, 'fixed_ip4_list': ['133.19.126.67']} 2025-11-27 04:13:02,231 - DEBUG - VM uve shows ip address as ['133.19.126.67', '::'] 2025-11-27 04:13:02,231 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 2025-11-27 04:13:02,231 - INFO - Verifying through opserver in 10.0.0.27 2025-11-27 04:13:02,231 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471?flat 2025-11-27 04:13:02,249 - DEBUG - expected vm list e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:02,249 - DEBUG - Extracted vm list ['e5d1d560-83b6-4bad-a40b-4d1f414bca54'] 2025-11-27 04:13:02,249 - INFO - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is present in default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 2025-11-27 04:13:02,249 - DEBUG - Verifying vm in vrouter uve 2025-11-27 04:13:02,249 - DEBUG - Getting info from collector 10.0.0.23.. 2025-11-27 04:13:02,249 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat 2025-11-27 04:13:02,311 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-4820-1'] 2025-11-27 04:13:02,311 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:02,368 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:02,400 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:02,400 - DEBUG - Validated that VM ctest-vm1-76104767 is in Vrouter an-jenkins-deploy-platform-ansible-os-4820-1 UVE 2025-11-27 04:13:02,400 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2025-11-27 04:13:02,409 - DEBUG - Expected tap interface of VM uuid e5d1d560-83b6-4bad-a40b-4d1f414bca54 is default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e 2025-11-27 04:13:02,409 - DEBUG - Expected VN of VM uuid e5d1d560-83b6-4bad-a40b-4d1f414bca54 is default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 2025-11-27 04:13:02,410 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:02,465 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:02,496 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:02,496 - INFO - tap interface default-domain:ctest-TestDropStats-65491859:3156545a-3851-4915-a806-30efc0fdb36e of vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:02,496 - DEBUG - Connected network default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 of vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:02,496 - DEBUG - Verifying the e5d1d560-83b6-4bad-a40b-4d1f414bca54 virtual network link through opserver 10.0.0.27 2025-11-27 04:13:02,496 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-11-27 04:13:02,512 - DEBUG - VM link and name as {'name': 'e5d1d560-83b6-4bad-a40b-4d1f414bca54', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat'} 2025-11-27 04:13:02,512 - DEBUG - Verifying the e5d1d560-83b6-4bad-a40b-4d1f414bca54 virtual network link through opserver 10.0.0.33 2025-11-27 04:13:02,512 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-11-27 04:13:02,826 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-11-27 04:13:02,904 - DEBUG - VM link and name as {'name': 'e5d1d560-83b6-4bad-a40b-4d1f414bca54', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat'} 2025-11-27 04:13:02,904 - DEBUG - Verifying the e5d1d560-83b6-4bad-a40b-4d1f414bca54 virtual network link through opserver 10.0.0.23 2025-11-27 04:13:02,904 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-11-27 04:13:02,916 - DEBUG - VM link and name as {'name': 'e5d1d560-83b6-4bad-a40b-4d1f414bca54', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54?flat'} 2025-11-27 04:13:02,916 - INFO - VM ctest-vm1-76104767 validations in Opserver passed 2025-11-27 04:13:02,997 - DEBUG - VM is in ACTIVE state now 2025-11-27 04:13:02,997 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-27 04:13:02,997 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:03,006 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/87e01ad3-38b3-4628-992a-c95f3ccca740 2025-11-27 04:13:03,017 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:03,025 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/87e01ad3-38b3-4628-992a-c95f3ccca740 2025-11-27 04:13:03,036 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/609a55b9-2b51-4a63-98af-5cd745c0f2f7 2025-11-27 04:13:03,047 - DEBUG - Verifying in api server 10.0.0.23 2025-11-27 04:13:03,047 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:03,055 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/87e01ad3-38b3-4628-992a-c95f3ccca740 2025-11-27 04:13:03,068 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/609a55b9-2b51-4a63-98af-5cd745c0f2f7 2025-11-27 04:13:03,076 - INFO - VM ctest-vm2-38853728 verfication in all API Servers passed 2025-11-27 04:13:03,094 - DEBUG - VM ctest-vm2-38853728 Tap interface: {'index': '3', 'name': 'tap87e01ad3-38', 'uuid': '87e01ad3-38b3-4628-992a-c95f3ccca740', 'vrf_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138:ctest-vn-69183138', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '24', 'l2_label': '29', 'vxlan_id': '7', 'vn_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138', 'vm_uuid': '1bc0c4b3-83e4-4950-8e41-e0892064fb15', 'vm_name': 'ctest-vm2-38853728', 'ip_addr': '171.235.56.67', 'mac_addr': '02:87:e0:1a:d3:38', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '168', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740', 'sg_uuid_list': ['7b76b989-6e3d-4669-b9f8-b2c2b64d0094'], 'static_route_list': None, 'vm_project_uuid': '367ce5ba-6bb9-4dd4-afb8-b1ca7effca72', 'admin_state': 'Enabled', 'flow_key_idx': '23', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-11-27 04:13:03,094 - DEBUG - Agent 10.0.0.130 vrf name: default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138:ctest-vn-69183138 2025-11-27 04:13:03,098 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138:ctest-vn-69183138', 'ucindex': '2', 'mcindex': '2', 'l2index': '2', 'source': 'Config; ', 'uc6index': '2', 'vn': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138', 'table_label': '-1', 'vxlan_id': '7', 'evpnindex': '2', 'brindex': '2', 'mplsindex': '0', 'RD': '10.20.0.200:2', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2025-11-27 04:13:03,179 - DEBUG - VM ctest-vm2-38853728 : Tap interface tap87e01ad3-38 is set to Active 2025-11-27 04:13:03,179 - DEBUG - Tap interface tap87e01ad3-38 detail : {'index': '3', 'name': 'tap87e01ad3-38', 'uuid': '87e01ad3-38b3-4628-992a-c95f3ccca740', 'vrf_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138:ctest-vn-69183138', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '24', 'l2_label': '29', 'vxlan_id': '7', 'vn_name': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138', 'vm_uuid': '1bc0c4b3-83e4-4950-8e41-e0892064fb15', 'vm_name': 'ctest-vm2-38853728', 'ip_addr': '171.235.56.67', 'mac_addr': '02:87:e0:1a:d3:38', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '168', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740', 'sg_uuid_list': ['7b76b989-6e3d-4669-b9f8-b2c2b64d0094'], 'static_route_list': None, 'vm_project_uuid': '367ce5ba-6bb9-4dd4-afb8-b1ca7effca72', 'admin_state': 'Enabled', 'flow_key_idx': '23', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-11-27 04:13:03,294 - DEBUG - Starting Layer 2 verification in Agent 2025-11-27 04:13:03,308 - DEBUG - Layer 2 path is seen for VM MAC 02:87:e0:1a:d3:38 in agent 10.0.0.130 2025-11-27 04:13:03,308 - DEBUG - Active layer 2 route in agent is present for VMI tap87e01ad3-38 2025-11-27 04:13:03,308 - DEBUG - L2 label(29) matches bw route table and interface table 2025-11-27 04:13:03,322 - DEBUG - VN default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 is found in Agent of node 10.0.0.130 2025-11-27 04:13:03,456 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 is consistent in agent 10.0.0.130 2025-11-27 04:13:03,456 - DEBUG - Route for VM IP ['171.235.56.67'] is consistent in agent 10.0.0.130 2025-11-27 04:13:03,456 - DEBUG - VN default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 verification for VM ctest-vm2-38853728 in Agent 10.0.0.130 passed 2025-11-27 04:13:03,578 - DEBUG - Starting all layer 2 verification in agent 10.0.0.130 2025-11-27 04:13:03,588 - DEBUG - Route for VM MAC 02:87:e0:1a:d3:38 is consistent in agent 10.0.0.130 2025-11-27 04:13:03,588 - INFO - VM ctest-vm2-38853728 verifications in Compute nodes passed 2025-11-27 04:13:03,827 - DEBUG - Validated VM route 171.235.56.67 in vrouter of 10.0.0.130 2025-11-27 04:13:03,832 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2025-11-27 04:13:03,832 - INFO - Validated routes of VM ctest-vm2-38853728 in all vrouters 2025-11-27 04:13:04,372 - DEBUG - Starting all layer2 verification in 10.0.0.27 Control Node 2025-11-27 04:13:04,495 - DEBUG - Layer2 route found for VM MAC 02:87:e0:1a:d3:38 in Control-node 10.0.0.27 2025-11-27 04:13:04,495 - DEBUG - L2 Label for VM ctest-vm2-38853728 same between Control-node 10.0.0.27 and Agent, Expected: 29, Seen: 29 2025-11-27 04:13:04,495 - DEBUG - Starting all layer2 verification in 10.0.0.33 Control Node 2025-11-27 04:13:04,622 - DEBUG - Layer2 route found for VM MAC 02:87:e0:1a:d3:38 in Control-node 10.0.0.33 2025-11-27 04:13:04,622 - DEBUG - L2 Label for VM ctest-vm2-38853728 same between Control-node 10.0.0.33 and Agent, Expected: 29, Seen: 29 2025-11-27 04:13:04,622 - INFO - Verification in Control-nodes for VM ctest-vm2-38853728 passed 2025-11-27 04:13:04,622 - DEBUG - Verifying the vm in opserver 2025-11-27 04:13:04,622 - DEBUG - Verifying in collector 10.0.0.23 ... 2025-11-27 04:13:04,622 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat 2025-11-27 04:13:04,636 - DEBUG - Failed to get VM ctest-vm2-38853728, ID 1bc0c4b3-83e4-4950-8e41-e0892064fb15 info from Opserver 2025-11-27 04:13:06,636 - DEBUG - Verifying the vm in opserver 2025-11-27 04:13:06,636 - DEBUG - Verifying in collector 10.0.0.23 ... 2025-11-27 04:13:06,636 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat 2025-11-27 04:13:06,653 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740?flat 2025-11-27 04:13:06,672 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740?flat 2025-11-27 04:13:06,690 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat 2025-11-27 04:13:06,704 - DEBUG - Verifying vm in vn uve 2025-11-27 04:13:06,704 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740?flat 2025-11-27 04:13:06,732 - DEBUG - VM uve shows interface as {'port_mirror_enabled': False, 'ip_address': '171.235.56.67', 'is_health_check_active': True, 'admin_state': True, 'vm_uuid': '1bc0c4b3-83e4-4950-8e41-e0892064fb15', 'rx_vlan': 65535, 'tx_vlan': 65535, 'label': 24, 'l2_active': True, 'virtual_network': 'default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138', 'vn_uuid': '92a36183-0312-42f4-b932-0fd99b5cc02f', 'vhostuser_mode': 0, 'vm_name': 'ctest-vm2-38853728', 'ip4_active': True, '__T': 1764216784798096, 'gateway': '171.235.56.65', 'active': True, 'ip6_address': '::', 'mac_address': '02:87:e0:1a:d3:38', 'uuid': '87e01ad3-38b3-4628-992a-c95f3ccca740', 'fixed_ip4_list': ['171.235.56.67'], 'ip6_active': False} 2025-11-27 04:13:06,732 - DEBUG - VM uve shows ip address as ['171.235.56.67', '::'] 2025-11-27 04:13:06,732 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 2025-11-27 04:13:06,732 - INFO - Verifying through opserver in 10.0.0.27 2025-11-27 04:13:06,732 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138?flat 2025-11-27 04:13:06,754 - DEBUG - expected vm list 1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:06,754 - DEBUG - Extracted vm list ['1bc0c4b3-83e4-4950-8e41-e0892064fb15'] 2025-11-27 04:13:06,754 - INFO - VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is present in default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 2025-11-27 04:13:06,754 - DEBUG - Verifying vm in vrouter uve 2025-11-27 04:13:06,754 - DEBUG - Getting info from collector 10.0.0.23.. 2025-11-27 04:13:06,754 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat 2025-11-27 04:13:06,770 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-4820-2'] 2025-11-27 04:13:06,770 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-2:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:06,819 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-2?flat 2025-11-27 04:13:06,897 - DEBUG - VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:06,897 - DEBUG - Validated that VM ctest-vm2-38853728 is in Vrouter an-jenkins-deploy-platform-ansible-os-4820-2 UVE 2025-11-27 04:13:06,897 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2025-11-27 04:13:06,903 - DEBUG - Expected tap interface of VM uuid 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740 2025-11-27 04:13:06,904 - DEBUG - Expected VN of VM uuid 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 2025-11-27 04:13:06,904 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-2:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:06,951 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-2?flat 2025-11-27 04:13:07,028 - DEBUG - VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:07,028 - INFO - tap interface default-domain:ctest-TestDropStats-65491859:87e01ad3-38b3-4628-992a-c95f3ccca740 of vm 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:07,028 - DEBUG - Connected network default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 of vm 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:07,028 - DEBUG - Verifying the 1bc0c4b3-83e4-4950-8e41-e0892064fb15 virtual network link through opserver 10.0.0.27 2025-11-27 04:13:07,028 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-11-27 04:13:07,040 - DEBUG - VM link and name as {'name': '1bc0c4b3-83e4-4950-8e41-e0892064fb15', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat'} 2025-11-27 04:13:07,040 - DEBUG - Verifying the 1bc0c4b3-83e4-4950-8e41-e0892064fb15 virtual network link through opserver 10.0.0.33 2025-11-27 04:13:07,040 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-11-27 04:13:07,052 - DEBUG - VM link and name as {'name': '1bc0c4b3-83e4-4950-8e41-e0892064fb15', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat'} 2025-11-27 04:13:07,052 - DEBUG - Verifying the 1bc0c4b3-83e4-4950-8e41-e0892064fb15 virtual network link through opserver 10.0.0.23 2025-11-27 04:13:07,053 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-11-27 04:13:07,066 - DEBUG - VM link and name as {'name': '1bc0c4b3-83e4-4950-8e41-e0892064fb15', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15?flat'} 2025-11-27 04:13:07,067 - INFO - VM ctest-vm2-38853728 validations in Opserver passed 2025-11-27 04:13:07,089 - INFO - ---------------------------------------------------------------------------------------------------- 2025-11-27 04:13:07,089 - INFO - ds_discard : 3 2025-11-27 04:13:07,089 - INFO - ds_pull : 0 2025-11-27 04:13:07,089 - INFO - ds_invalid_if : 2 2025-11-27 04:13:07,089 - INFO - ds_invalid_arp : 0 2025-11-27 04:13:07,089 - INFO - ds_trap_no_if : 0 2025-11-27 04:13:07,089 - INFO - ds_nowhere_to_go : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_queue_limit_exceeded : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_no_memory : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_invalid_protocol : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_nat_no_rflow : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_action_drop : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_action_invalid : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_unusable : 0 2025-11-27 04:13:07,089 - INFO - ds_flow_table_full : 0 2025-11-27 04:13:07,089 - INFO - ds_interface_tx_discard : 0 2025-11-27 04:13:07,090 - INFO - ds_interface_drop : 0 2025-11-27 04:13:07,090 - INFO - ds_duplicated : 0 2025-11-27 04:13:07,090 - INFO - ds_push : 0 2025-11-27 04:13:07,090 - INFO - ds_ttl_exceeded : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_nh : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_label : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_protocol : 0 2025-11-27 04:13:07,090 - INFO - ds_interface_rx_discard : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_mcast_source : 0 2025-11-27 04:13:07,090 - INFO - ds_head_alloc_fail : 0 2025-11-27 04:13:07,090 - INFO - ds_pcow_fail : 0 2025-11-27 04:13:07,090 - INFO - ds_mcast_clone_fail : 0 2025-11-27 04:13:07,090 - INFO - ds_rewrite_fail : 0 2025-11-27 04:13:07,090 - INFO - ds_misc : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_packet : 0 2025-11-27 04:13:07,090 - INFO - ds_cksum_err : 0 2025-11-27 04:13:07,090 - INFO - ds_no_fmd : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_vnid : 0 2025-11-27 04:13:07,090 - INFO - ds_frag_err : 0 2025-11-27 04:13:07,090 - INFO - ds_invalid_source : 0 2025-11-27 04:13:07,090 - INFO - ds_mcast_df_bit : 0 2025-11-27 04:13:07,091 - INFO - ds_l2_no_route : 0 2025-11-27 04:13:07,091 - INFO - ds_vlan_fwd_tx : 0 2025-11-27 04:13:07,091 - INFO - ds_vlan_fwd_enq : 0 2025-11-27 04:13:07,091 - INFO - ds_drop_new_flow : 0 2025-11-27 04:13:07,091 - INFO - ds_flow_evict : 0 2025-11-27 04:13:07,091 - INFO - ds_trap_original : 0 2025-11-27 04:13:07,091 - INFO - ds_no_memory : 0 2025-11-27 04:13:07,091 - INFO - ds_fragment_queue_fail : 0 2025-11-27 04:13:07,091 - INFO - ds_no_frag_entry : 0 2025-11-27 04:13:07,091 - INFO - ds_icmp_error : 0 2025-11-27 04:13:07,091 - INFO - ds_clone_fail : 0 2025-11-27 04:13:07,091 - INFO - ds_drop_pkts : 5 2025-11-27 04:13:07,102 - INFO - ---------------------------------------------------------------------------------------------------- 2025-11-27 04:13:07,102 - INFO - ds_discard : 6 2025-11-27 04:13:07,102 - INFO - ds_pull : 0 2025-11-27 04:13:07,102 - INFO - ds_invalid_if : 2 2025-11-27 04:13:07,102 - INFO - ds_invalid_arp : 0 2025-11-27 04:13:07,102 - INFO - ds_trap_no_if : 0 2025-11-27 04:13:07,102 - INFO - ds_nowhere_to_go : 0 2025-11-27 04:13:07,102 - INFO - ds_flow_queue_limit_exceeded : 0 2025-11-27 04:13:07,103 - INFO - ds_flow_no_memory : 0 2025-11-27 04:13:07,103 - INFO - ds_flow_invalid_protocol : 0 2025-11-27 04:13:07,103 - INFO - ds_flow_nat_no_rflow : 0 2025-11-27 04:13:07,103 - INFO - ds_flow_action_drop : 16 2025-11-27 04:13:07,103 - INFO - ds_flow_action_invalid : 0 2025-11-27 04:13:07,103 - INFO - ds_flow_unusable : 0 2025-11-27 04:13:07,103 - INFO - ds_flow_table_full : 0 2025-11-27 04:13:07,103 - INFO - ds_interface_tx_discard : 0 2025-11-27 04:13:07,103 - INFO - ds_interface_drop : 0 2025-11-27 04:13:07,103 - INFO - ds_duplicated : 1 2025-11-27 04:13:07,103 - INFO - ds_push : 0 2025-11-27 04:13:07,103 - INFO - ds_ttl_exceeded : 0 2025-11-27 04:13:07,103 - INFO - ds_invalid_nh : 4 2025-11-27 04:13:07,103 - INFO - ds_invalid_label : 0 2025-11-27 04:13:07,103 - INFO - ds_invalid_protocol : 0 2025-11-27 04:13:07,103 - INFO - ds_interface_rx_discard : 0 2025-11-27 04:13:07,103 - INFO - ds_invalid_mcast_source : 0 2025-11-27 04:13:07,103 - INFO - ds_head_alloc_fail : 0 2025-11-27 04:13:07,103 - INFO - ds_pcow_fail : 0 2025-11-27 04:13:07,103 - INFO - ds_mcast_clone_fail : 0 2025-11-27 04:13:07,103 - INFO - ds_rewrite_fail : 0 2025-11-27 04:13:07,103 - INFO - ds_misc : 0 2025-11-27 04:13:07,103 - INFO - ds_invalid_packet : 0 2025-11-27 04:13:07,103 - INFO - ds_cksum_err : 0 2025-11-27 04:13:07,103 - INFO - ds_no_fmd : 0 2025-11-27 04:13:07,103 - INFO - ds_invalid_vnid : 0 2025-11-27 04:13:07,104 - INFO - ds_frag_err : 0 2025-11-27 04:13:07,104 - INFO - ds_invalid_source : 0 2025-11-27 04:13:07,104 - INFO - ds_mcast_df_bit : 0 2025-11-27 04:13:07,104 - INFO - ds_l2_no_route : 2 2025-11-27 04:13:07,104 - INFO - ds_vlan_fwd_tx : 0 2025-11-27 04:13:07,104 - INFO - ds_vlan_fwd_enq : 0 2025-11-27 04:13:07,104 - INFO - ds_drop_new_flow : 0 2025-11-27 04:13:07,104 - INFO - ds_flow_evict : 0 2025-11-27 04:13:07,104 - INFO - ds_trap_original : 0 2025-11-27 04:13:07,104 - INFO - ds_no_memory : 0 2025-11-27 04:13:07,104 - INFO - ds_fragment_queue_fail : 0 2025-11-27 04:13:07,104 - INFO - ds_no_frag_entry : 0 2025-11-27 04:13:07,104 - INFO - ds_icmp_error : 0 2025-11-27 04:13:07,104 - INFO - ds_clone_fail : 0 2025-11-27 04:13:07,104 - INFO - ds_drop_pkts : 31 2025-11-27 04:13:07,104 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 5 -W 1 171.235.56.67, host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.19, gateway password: c0ntrail123 2025-11-27 04:13:07,104 - DEBUG - ping -s 56 -c 5 -W 1 171.235.56.67 2025-11-27 04:13:13,137 - DEBUG - PING 171.235.56.67 (171.235.56.67): 56 data bytes --- 171.235.56.67 ping statistics --- 5 packets transmitted, 0 packets received, 100% packet loss 2025-11-27 04:13:13,137 - WARNING - Ping to IP 171.235.56.67 from VM ctest-vm1-76104767 failed 2025-11-27 04:13:27,252 - INFO - Vif and vrouter dropstats of type ds_flow_action_drop verifed 2025-11-27 04:13:27,252 - INFO - ---------------------------------------------------------------------------------------------------- 2025-11-27 04:13:27,252 - INFO - ds_discard : 3 2025-11-27 04:13:27,252 - INFO - ds_pull : 0 2025-11-27 04:13:27,252 - INFO - ds_invalid_if : 2 2025-11-27 04:13:27,252 - INFO - ds_invalid_arp : 0 2025-11-27 04:13:27,252 - INFO - ds_trap_no_if : 0 2025-11-27 04:13:27,252 - INFO - ds_nowhere_to_go : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_queue_limit_exceeded : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_no_memory : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_invalid_protocol : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_nat_no_rflow : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_action_drop : 5 2025-11-27 04:13:27,252 - INFO - ds_flow_action_invalid : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_unusable : 0 2025-11-27 04:13:27,252 - INFO - ds_flow_table_full : 0 2025-11-27 04:13:27,252 - INFO - ds_interface_tx_discard : 0 2025-11-27 04:13:27,253 - INFO - ds_interface_drop : 0 2025-11-27 04:13:27,253 - INFO - ds_duplicated : 0 2025-11-27 04:13:27,253 - INFO - ds_push : 0 2025-11-27 04:13:27,253 - INFO - ds_ttl_exceeded : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_nh : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_label : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_protocol : 0 2025-11-27 04:13:27,253 - INFO - ds_interface_rx_discard : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_mcast_source : 0 2025-11-27 04:13:27,253 - INFO - ds_head_alloc_fail : 0 2025-11-27 04:13:27,253 - INFO - ds_pcow_fail : 0 2025-11-27 04:13:27,253 - INFO - ds_mcast_clone_fail : 0 2025-11-27 04:13:27,253 - INFO - ds_rewrite_fail : 0 2025-11-27 04:13:27,253 - INFO - ds_misc : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_packet : 0 2025-11-27 04:13:27,253 - INFO - ds_cksum_err : 0 2025-11-27 04:13:27,253 - INFO - ds_no_fmd : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_vnid : 0 2025-11-27 04:13:27,253 - INFO - ds_frag_err : 0 2025-11-27 04:13:27,253 - INFO - ds_invalid_source : 0 2025-11-27 04:13:27,253 - INFO - ds_mcast_df_bit : 0 2025-11-27 04:13:27,253 - INFO - ds_l2_no_route : 0 2025-11-27 04:13:27,253 - INFO - ds_vlan_fwd_tx : 0 2025-11-27 04:13:27,253 - INFO - ds_vlan_fwd_enq : 0 2025-11-27 04:13:27,253 - INFO - ds_drop_new_flow : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_evict : 0 2025-11-27 04:13:27,254 - INFO - ds_trap_original : 0 2025-11-27 04:13:27,254 - INFO - ds_no_memory : 0 2025-11-27 04:13:27,254 - INFO - ds_fragment_queue_fail : 0 2025-11-27 04:13:27,254 - INFO - ds_no_frag_entry : 0 2025-11-27 04:13:27,254 - INFO - ds_icmp_error : 0 2025-11-27 04:13:27,254 - INFO - ds_clone_fail : 0 2025-11-27 04:13:27,254 - INFO - ds_drop_pkts : 10 2025-11-27 04:13:27,254 - INFO - ---------------------------------------------------------------------------------------------------- 2025-11-27 04:13:27,254 - INFO - ds_discard : 6 2025-11-27 04:13:27,254 - INFO - ds_pull : 0 2025-11-27 04:13:27,254 - INFO - ds_invalid_if : 2 2025-11-27 04:13:27,254 - INFO - ds_invalid_arp : 0 2025-11-27 04:13:27,254 - INFO - ds_trap_no_if : 0 2025-11-27 04:13:27,254 - INFO - ds_nowhere_to_go : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_queue_limit_exceeded : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_no_memory : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_invalid_protocol : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_nat_no_rflow : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_action_drop : 21 2025-11-27 04:13:27,254 - INFO - ds_flow_action_invalid : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_unusable : 0 2025-11-27 04:13:27,254 - INFO - ds_flow_table_full : 0 2025-11-27 04:13:27,254 - INFO - ds_interface_tx_discard : 0 2025-11-27 04:13:27,254 - INFO - ds_interface_drop : 0 2025-11-27 04:13:27,254 - INFO - ds_duplicated : 1 2025-11-27 04:13:27,254 - INFO - ds_push : 0 2025-11-27 04:13:27,255 - INFO - ds_ttl_exceeded : 0 2025-11-27 04:13:27,255 - INFO - ds_invalid_nh : 4 2025-11-27 04:13:27,255 - INFO - ds_invalid_label : 0 2025-11-27 04:13:27,255 - INFO - ds_invalid_protocol : 0 2025-11-27 04:13:27,255 - INFO - ds_interface_rx_discard : 0 2025-11-27 04:13:27,255 - INFO - ds_invalid_mcast_source : 0 2025-11-27 04:13:27,255 - INFO - ds_head_alloc_fail : 0 2025-11-27 04:13:27,255 - INFO - ds_pcow_fail : 0 2025-11-27 04:13:27,255 - INFO - ds_mcast_clone_fail : 0 2025-11-27 04:13:27,255 - INFO - ds_rewrite_fail : 0 2025-11-27 04:13:27,255 - INFO - ds_misc : 0 2025-11-27 04:13:27,255 - INFO - ds_invalid_packet : 0 2025-11-27 04:13:27,255 - INFO - ds_cksum_err : 0 2025-11-27 04:13:27,255 - INFO - ds_no_fmd : 0 2025-11-27 04:13:27,255 - INFO - ds_invalid_vnid : 0 2025-11-27 04:13:27,255 - INFO - ds_frag_err : 0 2025-11-27 04:13:27,255 - INFO - ds_invalid_source : 0 2025-11-27 04:13:27,255 - INFO - ds_mcast_df_bit : 0 2025-11-27 04:13:27,255 - INFO - ds_l2_no_route : 2 2025-11-27 04:13:27,255 - INFO - ds_vlan_fwd_tx : 0 2025-11-27 04:13:27,255 - INFO - ds_vlan_fwd_enq : 0 2025-11-27 04:13:27,255 - INFO - ds_drop_new_flow : 0 2025-11-27 04:13:27,255 - INFO - ds_flow_evict : 0 2025-11-27 04:13:27,256 - INFO - ds_trap_original : 0 2025-11-27 04:13:27,256 - INFO - ds_no_memory : 0 2025-11-27 04:13:27,256 - INFO - ds_fragment_queue_fail : 0 2025-11-27 04:13:27,256 - INFO - ds_no_frag_entry : 0 2025-11-27 04:13:27,256 - INFO - ds_icmp_error : 0 2025-11-27 04:13:27,256 - INFO - ds_clone_fail : 0 2025-11-27 04:13:27,256 - INFO - ds_drop_pkts : 36 2025-11-27 04:13:27,256 - INFO - Deleting VM ctest-vm2-38853728 2025-11-27 04:13:27,325 - DEBUG - Verifying in api server 10.0.0.23 2025-11-27 04:13:27,326 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:27,334 - DEBUG - VM ID 1bc0c4b3-83e4-4950-8e41-e0892064fb15 of VM ctest-vm2-38853728 is still found in API Server 2025-11-27 04:13:29,335 - DEBUG - Verifying in api server 10.0.0.23 2025-11-27 04:13:29,335 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:29,342 - DEBUG - Response Code: 404 2025-11-27 04:13:29,343 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:29,350 - DEBUG - Response Code: 404 2025-11-27 04:13:29,350 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/1bc0c4b3-83e4-4950-8e41-e0892064fb15 2025-11-27 04:13:29,356 - DEBUG - Response Code: 404 2025-11-27 04:13:29,357 - INFO - VM ctest-vm2-38853728 is fully removed in API-Server 2025-11-27 04:13:29,507 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2025-11-27 04:13:29,551 - DEBUG - Validated that vrouter 10.0.0.19 does not have VMs route for VN default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 2025-11-27 04:13:29,563 - DEBUG - Validated that vrouter 10.0.0.130 does not have VMs route for VN default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 2025-11-27 04:13:29,563 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 2025-11-27 04:13:29,563 - INFO - VM ctest-vm2-38853728 is removed in Compute, and routes are removed in all compute nodes 2025-11-27 04:13:29,790 - INFO - Routes for VM ctest-vm2-38853728 is removed in all control-nodes 2025-11-27 04:13:29,790 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-2:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:29,842 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-2?flat 2025-11-27 04:13:29,875 - DEBUG - VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:29,875 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:29,875 - DEBUG - interface for vm 1bc0c4b3-83e4-4950-8e41-e0892064fb15 deleted from vrouter uve ... 2025-11-27 04:13:29,875 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:32,875 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-2:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:32,981 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-2?flat 2025-11-27 04:13:33,013 - DEBUG - VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:33,013 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:33,013 - DEBUG - interface for vm 1bc0c4b3-83e4-4950-8e41-e0892064fb15 deleted from vrouter uve ... 2025-11-27 04:13:33,013 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:36,014 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-2:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:36,055 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-2?flat 2025-11-27 04:13:36,092 - DEBUG - VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is not present in vrouter an-jenkins-deploy-platform-ansible-os-4820-2 uve 2025-11-27 04:13:36,092 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:36,092 - DEBUG - interface for vm 1bc0c4b3-83e4-4950-8e41-e0892064fb15 deleted from vrouter uve ... 2025-11-27 04:13:36,092 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:36,092 - DEBUG - Validated that VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is removed in Vrouter UVE 2025-11-27 04:13:36,092 - DEBUG - Verifying through opserver in 10.0.0.27 2025-11-27 04:13:36,092 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138?flat 2025-11-27 04:13:36,108 - DEBUG - 1bc0c4b3-83e4-4950-8e41-e0892064fb15 BM not in default-domain:ctest-TestDropStats-65491859:ctest-vn-69183138 uve 2025-11-27 04:13:36,108 - DEBUG - Validated that VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is not present in the VN UVE 2025-11-27 04:13:36,108 - DEBUG - Verifying the 1bc0c4b3-83e4-4950-8e41-e0892064fb15 virtual network link through opserver 10.0.0.27 2025-11-27 04:13:36,108 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-11-27 04:13:36,119 - INFO - 1bc0c4b3-83e4-4950-8e41-e0892064fb15 vm uve deleted from opserver 2025-11-27 04:13:36,119 - DEBUG - Verifying the 1bc0c4b3-83e4-4950-8e41-e0892064fb15 virtual network link through opserver 10.0.0.33 2025-11-27 04:13:36,119 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-11-27 04:13:36,133 - INFO - 1bc0c4b3-83e4-4950-8e41-e0892064fb15 vm uve deleted from opserver 2025-11-27 04:13:36,133 - DEBUG - Verifying the 1bc0c4b3-83e4-4950-8e41-e0892064fb15 virtual network link through opserver 10.0.0.23 2025-11-27 04:13:36,133 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-11-27 04:13:36,145 - INFO - 1bc0c4b3-83e4-4950-8e41-e0892064fb15 vm uve deleted from opserver 2025-11-27 04:13:36,145 - INFO - Validated that VM 1bc0c4b3-83e4-4950-8e41-e0892064fb15 is removed from Opserver 2025-11-27 04:13:36,146 - INFO - Deleting VM ctest-vm1-76104767 2025-11-27 04:13:36,219 - DEBUG - Verifying in api server 10.0.0.23 2025-11-27 04:13:36,219 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:36,227 - DEBUG - VM ID e5d1d560-83b6-4bad-a40b-4d1f414bca54 of VM ctest-vm1-76104767 is still found in API Server 2025-11-27 04:13:38,228 - DEBUG - Verifying in api server 10.0.0.23 2025-11-27 04:13:38,228 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:38,235 - DEBUG - Response Code: 404 2025-11-27 04:13:38,235 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:38,241 - DEBUG - Response Code: 404 2025-11-27 04:13:38,241 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:38,246 - DEBUG - Response Code: 404 2025-11-27 04:13:38,246 - INFO - VM ctest-vm1-76104767 is fully removed in API-Server 2025-11-27 04:13:38,404 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2025-11-27 04:13:38,438 - DEBUG - Validated that vrouter 10.0.0.19 does not have VMs route for VN default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 2025-11-27 04:13:38,450 - DEBUG - Validated that vrouter 10.0.0.130 does not have VMs route for VN default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 2025-11-27 04:13:38,450 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 2025-11-27 04:13:38,451 - INFO - VM ctest-vm1-76104767 is removed in Compute, and routes are removed in all compute nodes 2025-11-27 04:13:38,678 - INFO - Routes for VM ctest-vm1-76104767 is removed in all control-nodes 2025-11-27 04:13:38,678 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:38,718 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:38,748 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:38,748 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:38,748 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:38,748 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:41,749 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:41,799 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:41,875 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:41,875 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:41,875 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:41,875 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:44,876 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:44,931 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:44,957 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:44,957 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:44,957 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:44,958 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:47,958 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:48,001 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:48,026 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:48,026 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:48,026 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:48,026 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:51,026 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:51,082 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:51,109 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:51,109 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:51,109 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:51,109 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:54,110 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:54,179 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:54,205 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is still present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:54,205 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:54,205 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:54,205 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:57,206 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4820-1:Compute:contrail-vrouter-agent:0?flat 2025-11-27 04:13:57,250 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4820-1?flat 2025-11-27 04:13:57,280 - DEBUG - VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is not present in vrouter an-jenkins-deploy-platform-ansible-os-4820-1 uve 2025-11-27 04:13:57,280 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-11-27 04:13:57,280 - DEBUG - interface for vm e5d1d560-83b6-4bad-a40b-4d1f414bca54 deleted from vrouter uve ... 2025-11-27 04:13:57,280 - DEBUG - Deleted interface not in error interface list ... 2025-11-27 04:13:57,280 - DEBUG - Validated that VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is removed in Vrouter UVE 2025-11-27 04:13:57,280 - DEBUG - Verifying through opserver in 10.0.0.27 2025-11-27 04:13:57,280 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471?flat 2025-11-27 04:13:57,304 - DEBUG - e5d1d560-83b6-4bad-a40b-4d1f414bca54 BM not in default-domain:ctest-TestDropStats-65491859:ctest-vn-75402471 uve 2025-11-27 04:13:57,304 - DEBUG - Validated that VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is not present in the VN UVE 2025-11-27 04:13:57,304 - DEBUG - Verifying the e5d1d560-83b6-4bad-a40b-4d1f414bca54 virtual network link through opserver 10.0.0.27 2025-11-27 04:13:57,305 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-11-27 04:13:57,319 - DEBUG - No links retuned for e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:57,319 - INFO - e5d1d560-83b6-4bad-a40b-4d1f414bca54 vm uve deleted from opserver 2025-11-27 04:13:57,319 - DEBUG - Verifying the e5d1d560-83b6-4bad-a40b-4d1f414bca54 virtual network link through opserver 10.0.0.33 2025-11-27 04:13:57,319 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-11-27 04:13:57,333 - DEBUG - No links retuned for e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:57,333 - INFO - e5d1d560-83b6-4bad-a40b-4d1f414bca54 vm uve deleted from opserver 2025-11-27 04:13:57,333 - DEBUG - Verifying the e5d1d560-83b6-4bad-a40b-4d1f414bca54 virtual network link through opserver 10.0.0.23 2025-11-27 04:13:57,333 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-11-27 04:13:57,346 - DEBUG - No links retuned for e5d1d560-83b6-4bad-a40b-4d1f414bca54 2025-11-27 04:13:57,346 - INFO - e5d1d560-83b6-4bad-a40b-4d1f414bca54 vm uve deleted from opserver 2025-11-27 04:13:57,346 - INFO - Validated that VM e5d1d560-83b6-4bad-a40b-4d1f414bca54 is removed from Opserver 2025-11-27 04:13:57,346 - INFO - Deleting VN ctest-vn-69183138 2025-11-27 04:13:57,608 - DEBUG - Response for deleting network () 2025-11-27 04:13:57,609 - INFO - Deleting VN ctest-vn-75402471 2025-11-27 04:13:57,798 - DEBUG - Response for deleting network () 2025-11-27 04:13:59,078 - DEBUG - Skipping xmpp flap check
2025-11-27 04:13:59,078 - INFO - END TEST : test_flow_action_drop_stats : PASSED[0:01:18]
2025-11-27 04:13:59,078 - INFO - -------------------------------------------------------------------------------- 2025-11-27 04:13:59,836 - INFO - Deleted project: ctest-TestDropStats-65491859, ID : 367ce5ba-6bb9-4dd4-afb8-b1ca7effca72