2026-10-12 03:36:51,635 - INFO - Domain Default found not creating 2026-10-12 03:36:51,765 - INFO - Project ctest-DisablePolicyEcmpIpv6-15220833 not found, creating it 2026-10-12 03:36:52,220 - INFO - Created Project:ctest-DisablePolicyEcmpIpv6-15220833, ID : ad58e7e4-b820-4463-867b-f3d75362f5c1 2026-10-12 03:36:53,879 - INFO - ================================================================================ 2026-10-12 03:36:53,879 - INFO - STARTING TEST : test_ecmp_with_static_routes 2026-10-12 03:36:53,879 - INFO - TEST DESCRIPTION : Description: Verify disabling policy for ECMP routes with static routes on VM Steps: 1. launch 1 VN and launch 3 VMs in it. 2. create a static route for a new subnet prefix and add this on 2 VMIs. this will create 2 ECMP routes. 3. Disable the policy on all VMIs. 4. Now from 3rd VM send traffic to an IP from static route prefix 5. add new ECMP destinations and verify load is distributed to new destinations too 6. remove ECMP destinations and verify load is distributed to remaining destinations Pass criteria: 1. traffic should go through fine 2. flows should not be created 3. load should be distributed among ecmp routes. 2026-10-12 03:36:55,142 - DEBUG - Nothing to compare xmpp stats {'10.0.0.188': {'10.20.0.30': '0', '10.20.0.25': '0'}, '10.0.0.189': {'10.20.0.30': '0', '10.20.0.25': '0'}} with 2026-10-12 03:36:55,142 - INFO - Address family configured is dual 2026-10-12 03:36:55,142 - INFO - we are setting address family as v6 2026-10-12 03:36:55,142 - INFO - we are setting address family as v6 2026-10-12 03:36:55,142 - INFO - Initial checks done. Running the testcase now 2026-10-12 03:36:55,142 - INFO - 2026-10-12 03:36:55,161 - DEBUG - Services list from nova: [, , , ] 2026-10-12 03:36:55,161 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-6357-1', 'an-jenkins-deploy-platform-ansible-os-6357-2']} 2026-10-12 03:36:55,804 - DEBUG - Response for create_network : {'network': {'id': '3b292775-bb82-4c08-a9b4-10c31d313474', 'name': 'ctest-vn-24776020', 'tenant_id': 'ad58e7e4b8204463867bf3d75362f5c1', 'project_id': 'ad58e7e4b8204463867bf3d75362f5c1', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'tags': [], 'subnets': [], 'fq_name': ['default-domain', 'ctest-DisablePolicyEcmpIpv6-15220833', 'ctest-vn-24776020'], 'port_security_enabled': True, 'description': ''}} 2026-10-12 03:36:56,026 - DEBUG - Response for create_subnet : {'subnet': {'id': '6c163383-9c4e-4c13-84a5-2ec0e4afadae', 'name': '', 'tenant_id': 'ad58e7e4b8204463867bf3d75362f5c1', 'network_id': '3b292775-bb82-4c08-a9b4-10c31d313474', 'ip_version': 6, 'cidr': '289a:736f:6cd9:4eba:c424:eeb7::/96', 'allocation_pools': [{'start': '289a:736f:6cd9:4eba:c424:eeb7:0:2', 'end': '289a:736f:6cd9:4eba:c424:eeb7:ffff:fffe'}], 'gateway_ip': '289a:736f:6cd9:4eba:c424:eeb7:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '289a:736f:6cd9:4eba:c424:eeb7:0:2', 'tags': [], 'project_id': 'ad58e7e4b8204463867bf3d75362f5c1'}} 2026-10-12 03:36:56,291 - DEBUG - Response for create_subnet : {'subnet': {'id': 'fe7b5704-d505-4de3-be62-a6ac5b6fb8e3', 'name': '', 'tenant_id': 'ad58e7e4b8204463867bf3d75362f5c1', 'network_id': '3b292775-bb82-4c08-a9b4-10c31d313474', 'ip_version': 4, 'cidr': '112.141.22.192/26', 'allocation_pools': [{'start': '112.141.22.194', 'end': '112.141.22.254'}], 'gateway_ip': '112.141.22.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '112.141.22.194', 'tags': [], 'project_id': 'ad58e7e4b8204463867bf3d75362f5c1'}} 2026-10-12 03:36:56,309 - INFO - Created VN ctest-vn-24776020 2026-10-12 03:36:56,318 - DEBUG - VN ctest-vn-24776020 UUID is 3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:56,394 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-10-12 03:36:56,754 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-10-12 03:36:56,777 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e7595c33-d9f2-4727-95af-2f5d18bc40e3 2026-10-12 03:36:56,790 - DEBUG - Requesting: http://10.0.0.38:8082/project/ad58e7e4-b820-4463-867b-f3d75362f5c1 2026-10-12 03:36:56,886 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:56,898 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:56,906 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/865b5a03-117e-439c-8f1e-f5a5de1eeeec 2026-10-12 03:36:56,912 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/865b5a03-117e-439c-8f1e-f5a5de1eeeec 2026-10-12 03:36:56,918 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/68a52c06-6ca9-45e0-8e0c-91f9ac2e1743 2026-10-12 03:36:56,923 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-10-12 03:36:56,923 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:56,932 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/865b5a03-117e-439c-8f1e-f5a5de1eeeec 2026-10-12 03:36:56,944 - INFO - Verified VN network id 15 for VN 3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:56,944 - INFO - Verifications in API Server for VN ctest-vn-24776020 passed 2026-10-12 03:36:56,945 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:56,953 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/865b5a03-117e-439c-8f1e-f5a5de1eeeec 2026-10-12 03:36:56,958 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/865b5a03-117e-439c-8f1e-f5a5de1eeeec 2026-10-12 03:36:56,967 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/68a52c06-6ca9-45e0-8e0c-91f9ac2e1743 2026-10-12 03:36:56,981 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-DisablePolicyEcmpIpv6-15220833', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4262981908893420552', 'uuid-lslong': '12228417318412170356'}, 'enable': 'true', 'created': '2026-10-12T03:36:55', 'last-modified': '2026-10-12T03:36:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.752917'} 2026-10-12 03:36:56,988 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-10-12 03:36:56,996 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-DisablePolicyEcmpIpv6-15220833', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4262981908893420552', 'uuid-lslong': '12228417318412170356'}, 'enable': 'true', 'created': '2026-10-12T03:36:55', 'last-modified': '2026-10-12T03:36:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.757739'} 2026-10-12 03:36:57,002 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-10-12 03:36:57,008 - DEBUG - Control-node 10.0.0.191 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-DisablePolicyEcmpIpv6-15220833', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4262981908893420552', 'uuid-lslong': '12228417318412170356'}, 'enable': 'true', 'created': '2026-10-12T03:36:55', 'last-modified': '2026-10-12T03:36:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.782856'} 2026-10-12 03:36:57,014 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-10-12 03:36:57,014 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-24776020 passed 2026-10-12 03:36:57,015 - DEBUG - ====Verifying policy data for ctest-vn-24776020 in API_Server ====== 2026-10-12 03:36:57,015 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-10-12 03:36:57,019 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e7595c33-d9f2-4727-95af-2f5d18bc40e3 2026-10-12 03:36:57,029 - DEBUG - Requesting: http://10.0.0.38:8082/project/ad58e7e4-b820-4463-867b-f3d75362f5c1 2026-10-12 03:36:57,111 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/3b292775-bb82-4c08-a9b4-10c31d313474 2026-10-12 03:36:57,121 - DEBUG - =>VN ctest-vn-24776020 has no policy to be verified 2026-10-12 03:36:57,122 - DEBUG - Verifying the vn in opserver 2026-10-12 03:36:57,122 - DEBUG - Verifying the default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 virtual network link through opserver 10.0.0.38 2026-10-12 03:36:57,122 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2026-10-12 03:36:57,481 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2026-10-12 03:36:57,531 - DEBUG - vn link and name as {'name': 'default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020?flat'} 2026-10-12 03:36:57,531 - DEBUG - Verifying the default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 virtual network link through opserver 10.0.0.50 2026-10-12 03:36:57,531 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2026-10-12 03:36:57,889 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2026-10-12 03:36:57,932 - DEBUG - vn link and name as {'name': 'default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020?flat'} 2026-10-12 03:36:57,932 - DEBUG - Verifying the default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 virtual network link through opserver 10.0.0.191 2026-10-12 03:36:57,932 - DEBUG - Requesting: http://10.0.0.191:8081/analytics/uves/virtual-networks 2026-10-12 03:36:58,288 - DEBUG - Requesting: http://10.0.0.191:8081/analytics/uves/virtual-networks 2026-10-12 03:36:58,333 - DEBUG - vn link and name as {'name': 'default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020', 'href': 'http://10.0.0.191:8081/analytics/uves/virtual-network/default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020?flat'} 2026-10-12 03:36:58,333 - INFO - Validated that VN default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is found in opserver 2026-10-12 03:36:58,351 - DEBUG - Do not have enough data to verify VN in agent 2026-10-12 03:36:58,359 - DEBUG - VRF ids for VN ctest-vn-24776020: {} 2026-10-12 03:36:58,359 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-6357-1', 'an-jenkins-deploy-platform-ansible-os-6357-2']} 2026-10-12 03:36:58,744 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-6357-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-6357-1) 2026-10-12 03:36:58,744 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-6357-1', 'an-jenkins-deploy-platform-ansible-os-6357-2']} 2026-10-12 03:36:59,187 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-6357-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-6357-2) 2026-10-12 03:36:59,640 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-6357-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-6357-2) 2026-10-12 03:36:59,640 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-6357-1', 'an-jenkins-deploy-platform-ansible-os-6357-2']} 2026-10-12 03:37:00,062 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-6357-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-6357-1) 2026-10-12 03:37:00,063 - INFO - Waiting for VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 to be up.. 2026-10-12 03:37:00,115 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:05,212 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:10,282 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:15,356 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:20,425 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:25,498 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:30,576 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:35,645 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-10-12 03:37:40,724 - DEBUG - VM is in ACTIVE state now 2026-10-12 03:37:40,724 - INFO - VM name : ctest-DisablePolicyEcmpIpv6-15220833-32032098 2026-10-12 03:37:40,826 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 ID is 9cc30743-787c-4219-bf0f-ceb1bfede47a 2026-10-12 03:37:40,827 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 launched on Node an-jenkins-deploy-platform-ansible-os-6357-2 2026-10-12 03:37:40,925 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine/9cc30743-787c-4219-bf0f-ceb1bfede47a 2026-10-12 03:37:41,284 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine/9cc30743-787c-4219-bf0f-ceb1bfede47a 2026-10-12 03:37:41,311 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine-interface/0e67e102-7ad1-4d5c-848c-e6eb9f7871cb 2026-10-12 03:37:44,492 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1028ms') 2026-10-12 03:37:44,492 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:37:44,504 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:37:44,504 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:37:48,596 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2026-10-12 03:37:48,596 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:37:48,608 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:37:48,608 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:37:52,681 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms') 2026-10-12 03:37:52,681 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:37:52,693 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:37:52,693 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:37:56,779 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2026-10-12 03:37:56,779 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:37:56,792 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:37:56,792 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:38:00,876 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2026-10-12 03:38:00,876 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:38:00,888 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:38:00,888 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:38:04,974 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1027ms') 2026-10-12 03:38:04,974 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:38:04,988 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:38:04,988 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:38:09,066 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2026-10-12 03:38:09,066 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:38:09,079 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:38:09,079 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:38:13,165 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2026-10-12 03:38:13,165 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 failed! 2026-10-12 03:38:13,177 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 289a:736f:6cd9:4eba:c424:eeb7:0:1 and allocation pool is NOT set 2026-10-12 03:38:13,177 - DEBUG - Gateway for vn default-domain:ctest-DisablePolicyEcmpIpv6-15220833:ctest-vn-24776020 is 112.141.22.193 and allocation pool is NOT set 2026-10-12 03:38:15,239 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=8.85 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.630 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.630/4.740/8.850/4.110 ms') 2026-10-12 03:38:15,239 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 passed 2026-10-12 03:38:15,301 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:15,301 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-32032098, IP 289a:736f:6cd9:4eba:c424:eeb7:0:4, Port 22 2026-10-12 03:38:15,373 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-10-12 03:38:15,475 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 is NOT ready for SSH connections, VM status: ACTIVE 2026-10-12 03:38:20,476 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:20,476 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-32032098, IP 289a:736f:6cd9:4eba:c424:eeb7:0:4, Port 22 2026-10-12 03:38:20,548 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-10-12 03:38:20,617 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 is NOT ready for SSH connections, VM status: ACTIVE 2026-10-12 03:38:25,618 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:25,618 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-32032098, IP 289a:736f:6cd9:4eba:c424:eeb7:0:4, Port 22 2026-10-12 03:38:25,684 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-10-12 03:38:25,796 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 is NOT ready for SSH connections, VM status: ACTIVE 2026-10-12 03:38:30,797 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:30,797 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-32032098, IP 289a:736f:6cd9:4eba:c424:eeb7:0:4, Port 22 2026-10-12 03:38:30,864 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-10-12 03:38:30,940 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 is NOT ready for SSH connections, VM status: ACTIVE 2026-10-12 03:38:35,941 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:35,941 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-32032098, IP 289a:736f:6cd9:4eba:c424:eeb7:0:4, Port 22 2026-10-12 03:38:36,011 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-10-12 03:38:36,080 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 is NOT ready for SSH connections, VM status: ACTIVE 2026-10-12 03:38:41,080 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:41,081 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-32032098, IP 289a:736f:6cd9:4eba:c424:eeb7:0:4, Port 22 2026-10-12 03:38:41,247 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 is ready for SSH connections 2026-10-12 03:38:41,247 - INFO - Waiting for VM ctest-DisablePolicyEcmpIpv6-15220833-06021352 to be up.. 2026-10-12 03:38:41,317 - DEBUG - VM is in ACTIVE state now 2026-10-12 03:38:41,317 - INFO - VM name : ctest-DisablePolicyEcmpIpv6-15220833-06021352 2026-10-12 03:38:41,383 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-06021352 ID is a5bb43a1-b7b1-4551-a86f-404ccefc4e8a 2026-10-12 03:38:41,383 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-06021352 launched on Node an-jenkins-deploy-platform-ansible-os-6357-2 2026-10-12 03:38:41,447 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine/a5bb43a1-b7b1-4551-a86f-404ccefc4e8a 2026-10-12 03:38:41,456 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine-interface/5ae85f50-1e78-40fc-9e21-1d0dbafabcea 2026-10-12 03:38:42,617 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.82 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=4.45 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 3.819/4.132/4.445/0.313 ms') 2026-10-12 03:38:42,617 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-DisablePolicyEcmpIpv6-15220833-06021352 passed 2026-10-12 03:38:42,682 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:42,682 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-06021352, IP 289a:736f:6cd9:4eba:c424:eeb7:0:6, Port 22 2026-10-12 03:38:42,856 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-06021352 is ready for SSH connections 2026-10-12 03:38:42,856 - INFO - Waiting for VM ctest-DisablePolicyEcmpIpv6-15220833-99642127 to be up.. 2026-10-12 03:38:42,925 - DEBUG - VM is in ACTIVE state now 2026-10-12 03:38:42,925 - INFO - VM name : ctest-DisablePolicyEcmpIpv6-15220833-99642127 2026-10-12 03:38:42,990 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-99642127 ID is 9dc1b3a7-0601-4ec9-9703-11f315f498ea 2026-10-12 03:38:42,990 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-99642127 launched on Node an-jenkins-deploy-platform-ansible-os-6357-1 2026-10-12 03:38:43,055 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine/9dc1b3a7-0601-4ec9-9703-11f315f498ea 2026-10-12 03:38:43,062 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine-interface/73d31eff-6526-45c5-8697-d062ffabe889 2026-10-12 03:38:44,217 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=2.69 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.72 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.689/2.704/2.720/0.015 ms') 2026-10-12 03:38:44,218 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-DisablePolicyEcmpIpv6-15220833-99642127 passed 2026-10-12 03:38:44,281 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:44,281 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-99642127, IP 289a:736f:6cd9:4eba:c424:eeb7:0:3, Port 22 2026-10-12 03:38:44,453 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-99642127 is ready for SSH connections 2026-10-12 03:38:44,454 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:44,454 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep 289a:736f:6cd9:4eba:c424:eeb7:0:4 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:38:44,454 - DEBUG - ifconfig -a| grep 289a:736f:6cd9:4eba:c424:eeb7:0:4 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1 2026-10-12 03:38:51,677 - DEBUG - eth0 2026-10-12 03:38:51,677 - DEBUG - Running remote_cmd, Cmd : sysctl net.ipv6.conf.eth0.accept_dad=0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:38:51,677 - DEBUG - sysctl net.ipv6.conf.eth0.accept_dad=0 2026-10-12 03:38:51,997 - DEBUG - net.ipv6.conf.eth0.accept_dad = 0 2026-10-12 03:38:52,195 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-DisablePolicyEcmpIpv6-15220833', 'ctest-intf-rtb-51661426'](UUID a18d021a-1616-4f83-a180-71d5e87ed4a9), prefixes : ['2a81:48ad:d784:4cb7:2b2b:34e3::/96'] 2026-10-12 03:38:52,259 - INFO - Added intf route table a18d021a-1616-4f83-a180-71d5e87ed4a9 to port 0e67e102-7ad1-4d5c-848c-e6eb9f7871cb 2026-10-12 03:38:52,259 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:38:52,259 - DEBUG - ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1 2026-10-12 03:38:52,443 - DEBUG - eth0 2026-10-12 03:38:52,444 - DEBUG - Running remote_cmd, Cmd : ifconfig eth0 inet6 add 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:38:52,444 - DEBUG - ifconfig eth0 inet6 add 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 2026-10-12 03:38:52,713 - DEBUG - None 2026-10-12 03:38:52,713 - DEBUG - Running remote_cmd, Cmd : ifconfig -a, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:38:52,713 - DEBUG - ifconfig -a 2026-10-12 03:38:52,984 - DEBUG - eth0 Link encap:Ethernet HWaddr 02:0e:67:e1:02:7a inet addr:112.141.22.196 Bcast:112.141.22.255 Mask:255.255.255.192 inet6 addr: 289a:736f:6cd9:4eba:c424:eeb7:0:4/64 Scope:Global inet6 addr: fe80::e:67ff:fee1:27a/64 Scope:Link inet6 addr: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def/128 Scope:Global UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:419 errors:0 dropped:0 overruns:0 frame:0 TX packets:374 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:42347 (42.3 KB) TX bytes:35126 (35.1 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 2026-10-12 03:38:52,984 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:38:52,984 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep 289a:736f:6cd9:4eba:c424:eeb7:0:6 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:38:52,984 - DEBUG - ifconfig -a| grep 289a:736f:6cd9:4eba:c424:eeb7:0:6 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1 2026-10-12 03:39:31,676 - DEBUG - eth0 2026-10-12 03:39:31,676 - DEBUG - Running remote_cmd, Cmd : sysctl net.ipv6.conf.eth0.accept_dad=0, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:39:31,676 - DEBUG - sysctl net.ipv6.conf.eth0.accept_dad=0 2026-10-12 03:39:31,990 - DEBUG - net.ipv6.conf.eth0.accept_dad = 0 2026-10-12 03:39:32,199 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-DisablePolicyEcmpIpv6-15220833', 'ctest-intf-rtb-70052547'](UUID 1ad8f91a-85d0-4001-a9da-e5eb41543da8), prefixes : ['2a81:48ad:d784:4cb7:2b2b:34e3::/96'] 2026-10-12 03:39:32,259 - INFO - Added intf route table 1ad8f91a-85d0-4001-a9da-e5eb41543da8 to port 5ae85f50-1e78-40fc-9e21-1d0dbafabcea 2026-10-12 03:39:32,259 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:39:32,259 - DEBUG - ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1 2026-10-12 03:39:32,451 - DEBUG - eth0 2026-10-12 03:39:32,451 - DEBUG - Running remote_cmd, Cmd : ifconfig eth0 inet6 add 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:39:32,451 - DEBUG - ifconfig eth0 inet6 add 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 2026-10-12 03:39:32,700 - DEBUG - None 2026-10-12 03:39:32,700 - DEBUG - Running remote_cmd, Cmd : ifconfig -a, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.189, gateway password: c0ntrail123 2026-10-12 03:39:32,700 - DEBUG - ifconfig -a 2026-10-12 03:39:32,998 - DEBUG - eth0 Link encap:Ethernet HWaddr 02:5a:e8:5f:50:1e inet addr:112.141.22.198 Bcast:112.141.22.255 Mask:255.255.255.192 inet6 addr: 289a:736f:6cd9:4eba:c424:eeb7:0:6/64 Scope:Global inet6 addr: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def/128 Scope:Global inet6 addr: fe80::5a:e8ff:fe5f:501e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:431 errors:0 dropped:0 overruns:0 frame:0 TX packets:387 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:42583 (42.5 KB) TX bytes:35878 (35.8 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 2026-10-12 03:39:33,059 - INFO - Policy DISABLED on VMI 73d31eff-6526-45c5-8697-d062ffabe889 2026-10-12 03:39:33,119 - INFO - Policy DISABLED on VMI 0e67e102-7ad1-4d5c-848c-e6eb9f7871cb 2026-10-12 03:39:33,179 - INFO - Policy DISABLED on VMI 5ae85f50-1e78-40fc-9e21-1d0dbafabcea 2026-10-12 03:39:33,195 - INFO - ECMP routes found in agent 10.0.0.189, for prefix 2a81:48ad:d784:4cb7:2b2b:34e3::/96 2026-10-12 03:39:33,195 - INFO - Tap interface tap0e67e102-7a found in ecmp routes in agent 10.0.0.189 2026-10-12 03:39:33,196 - INFO - Tap interface tap5ae85f50-1e found in ecmp routes in agent 10.0.0.189 2026-10-12 03:39:33,210 - INFO - ECMP routes found in agent 10.0.0.189, for prefix 2a81:48ad:d784:4cb7:2b2b:34e3::/96 2026-10-12 03:39:33,288 - DEBUG - Executing command: sudo tcpdump -nni tap0e67e102-7a -U '(udp and src host 289a:736f:6cd9:4eba:c424:eeb7:0:3 and dst host 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def and dst port 11000)' -w /tmp/tap0e67e102-7a_ctest-random-31609886.pcap 2026-10-12 03:39:33,461 - DEBUG - Executing command: sudo tcpdump -nni tap5ae85f50-1e -U '(udp and src host 289a:736f:6cd9:4eba:c424:eeb7:0:3 and dst host 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def and dst port 11000)' -w /tmp/tap5ae85f50-1e_ctest-random-17901772.pcap 2026-10-12 03:39:33,561 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 45733 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 11000 < icmp_error, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:39:33,561 - DEBUG - dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 45733 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 11000 < icmp_error 2026-10-12 03:40:15,517 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.0010869 s, 92.0 kB/s 2026-10-12 03:40:15,517 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:15,531 - DEBUG - STDOUT: 0 2026-10-12 03:40:15,531 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:15,531 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:17,532 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:17,548 - DEBUG - STDOUT: 0 2026-10-12 03:40:17,549 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:17,549 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:19,549 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:19,563 - DEBUG - STDOUT: 0 2026-10-12 03:40:19,563 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:19,564 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:21,565 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:21,580 - DEBUG - STDOUT: 0 2026-10-12 03:40:21,581 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:21,581 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:23,581 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:23,596 - DEBUG - STDOUT: 0 2026-10-12 03:40:23,596 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:23,596 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:25,597 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:25,612 - DEBUG - STDOUT: 0 2026-10-12 03:40:25,612 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:25,612 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:27,613 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0e67e102-7a_ctest-random-31609886.pcap | grep -c length 2026-10-12 03:40:27,627 - DEBUG - STDOUT: 0 2026-10-12 03:40:27,627 - DEBUG - STDERR: reading from file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:27,627 - WARNING - No packets are found in tcpdump output file /tmp/tap0e67e102-7a_ctest-random-31609886.pcap but expected some packets 2026-10-12 03:40:27,627 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap5ae85f50-1e_ctest-random-17901772.pcap | grep -c length 2026-10-12 03:40:27,641 - DEBUG - STDOUT: 1 2026-10-12 03:40:27,641 - DEBUG - STDERR: reading from file /tmp/tap5ae85f50-1e_ctest-random-17901772.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:40:27,641 - INFO - 1 packets are found in tcpdump output as expected 2026-10-12 03:40:27,641 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-10-12 03:40:29,685 - INFO - Tcpdump verification on VM 289a:736f:6cd9:4eba:c424:eeb7:0:6 passed 2026-10-12 03:40:29,903 - DEBUG - Forward and reverse flows for match {'sip': '289a:736f:6cd9:4eba:c424:eeb7:0:3', 'dip': '2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def', 'sport': '45733', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0 2026-10-12 03:40:29,904 - DEBUG - Flow verification PASSED on node: 10.0.0.189 for VMs - Sender: 289a:736f:6cd9:4eba:c424:eeb7:0:3, Receiver: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, Flow count expected: 0 0, got: 0 0 2026-10-12 03:40:29,904 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-10-12 03:40:31,911 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-10-12 03:40:34,166 - DEBUG - Forward and reverse flows for match {'sip': '289a:736f:6cd9:4eba:c424:eeb7:0:3', 'dip': '2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def', 'sport': '45733', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0 2026-10-12 03:40:34,167 - DEBUG - Flow verification PASSED on node: 10.0.0.188 for VMs - Sender: 289a:736f:6cd9:4eba:c424:eeb7:0:3, Receiver: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, Flow count expected: 0 0, got: 0 0 2026-10-12 03:40:34,167 - INFO - Traffic verification for ECMP passed 2026-10-12 03:40:34,167 - INFO - Waiting for VM ctest-DisablePolicyEcmpIpv6-15220833-97155360 to be up.. 2026-10-12 03:40:34,236 - DEBUG - VM is in ACTIVE state now 2026-10-12 03:40:34,236 - INFO - VM name : ctest-DisablePolicyEcmpIpv6-15220833-97155360 2026-10-12 03:40:34,297 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-97155360 ID is 83c83848-3b26-4eef-b144-a3548876497a 2026-10-12 03:40:34,297 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-97155360 launched on Node an-jenkins-deploy-platform-ansible-os-6357-1 2026-10-12 03:40:34,364 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine/83c83848-3b26-4eef-b144-a3548876497a 2026-10-12 03:40:34,371 - DEBUG - Requesting: http://10.0.0.191:8082/virtual-machine-interface/aaa5fa9d-7143-4621-8409-4312ee4b820c 2026-10-12 03:40:35,512 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.50 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.331 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.331/1.916/3.501/1.585 ms') 2026-10-12 03:40:35,512 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-DisablePolicyEcmpIpv6-15220833-97155360 passed 2026-10-12 03:40:35,573 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:40:35,573 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-15220833-97155360, IP 289a:736f:6cd9:4eba:c424:eeb7:0:5, Port 22 2026-10-12 03:40:35,742 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-15220833-97155360 is ready for SSH connections 2026-10-12 03:40:35,803 - INFO - Policy DISABLED on VMI aaa5fa9d-7143-4621-8409-4312ee4b820c 2026-10-12 03:40:35,803 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-10-12 03:40:35,803 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep 289a:736f:6cd9:4eba:c424:eeb7:0:5 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:40:35,803 - DEBUG - ifconfig -a| grep 289a:736f:6cd9:4eba:c424:eeb7:0:5 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1 2026-10-12 03:41:14,241 - DEBUG - eth0 2026-10-12 03:41:14,241 - DEBUG - Running remote_cmd, Cmd : sysctl net.ipv6.conf.eth0.accept_dad=0, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:41:14,242 - DEBUG - sysctl net.ipv6.conf.eth0.accept_dad=0 2026-10-12 03:41:14,551 - DEBUG - net.ipv6.conf.eth0.accept_dad = 0 2026-10-12 03:41:14,763 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-DisablePolicyEcmpIpv6-15220833', 'ctest-intf-rtb-09198052'](UUID 8ed0ff0e-b402-4fba-81a5-ee7fd1d955c3), prefixes : ['2a81:48ad:d784:4cb7:2b2b:34e3::/96'] 2026-10-12 03:41:14,827 - INFO - Added intf route table 8ed0ff0e-b402-4fba-81a5-ee7fd1d955c3 to port aaa5fa9d-7143-4621-8409-4312ee4b820c 2026-10-12 03:41:14,827 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:41:14,827 - DEBUG - ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1 2026-10-12 03:41:15,027 - DEBUG - eth0 2026-10-12 03:41:15,027 - DEBUG - Running remote_cmd, Cmd : ifconfig eth0 inet6 add 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:41:15,027 - DEBUG - ifconfig eth0 inet6 add 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 2026-10-12 03:41:15,278 - DEBUG - None 2026-10-12 03:41:15,279 - DEBUG - Running remote_cmd, Cmd : ifconfig -a, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:41:15,279 - DEBUG - ifconfig -a 2026-10-12 03:41:15,541 - DEBUG - eth0 Link encap:Ethernet HWaddr 02:aa:a5:fa:9d:71 inet addr:112.141.22.197 Bcast:112.141.22.255 Mask:255.255.255.192 inet6 addr: 289a:736f:6cd9:4eba:c424:eeb7:0:5/64 Scope:Global inet6 addr: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def/128 Scope:Global inet6 addr: fe80::aa:a5ff:fefa:9d71/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:573 errors:0 dropped:0 overruns:0 frame:0 TX packets:525 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:51741 (51.7 KB) TX bytes:45021 (45.0 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 2026-10-12 03:41:15,542 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-15220833-32032098 2026-10-12 03:41:15,671 - INFO - Removed intf route table a18d021a-1616-4f83-a180-71d5e87ed4a9 from port 0e67e102-7ad1-4d5c-848c-e6eb9f7871cb 2026-10-12 03:41:15,705 - INFO - Deleted Interface route table a18d021a-1616-4f83-a180-71d5e87ed4a9 2026-10-12 03:41:15,720 - INFO - ECMP routes found in agent 10.0.0.189, for prefix 2a81:48ad:d784:4cb7:2b2b:34e3::/96 2026-10-12 03:41:15,720 - INFO - Tap interface tap5ae85f50-1e found in ecmp routes in agent 10.0.0.189 2026-10-12 03:41:15,736 - INFO - ECMP routes found in agent 10.0.0.188, for prefix 2a81:48ad:d784:4cb7:2b2b:34e3::/96 2026-10-12 03:41:15,736 - INFO - Tap interface tapaaa5fa9d-71 found in ecmp routes in agent 10.0.0.188 2026-10-12 03:41:15,820 - DEBUG - Executing command: sudo tcpdump -nni tap5ae85f50-1e -U '(udp and src host 289a:736f:6cd9:4eba:c424:eeb7:0:3 and dst host 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def and dst port 11000)' -w /tmp/tap5ae85f50-1e_ctest-random-42862084.pcap 2026-10-12 03:41:16,288 - DEBUG - Executing command: sudo tcpdump -nni tapaaa5fa9d-71 -U '(udp and src host 289a:736f:6cd9:4eba:c424:eeb7:0:3 and dst host 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def and dst port 11000)' -w /tmp/tapaaa5fa9d-71_ctest-random-71243257.pcap 2026-10-12 03:41:16,372 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 20202 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 11000 < icmp_error, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.188, gateway password: c0ntrail123 2026-10-12 03:41:16,372 - DEBUG - dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 20202 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def 11000 < icmp_error 2026-10-12 03:41:18,609 - DEBUG - 1+0 records in 1+0 records out 100 bytes (100 B) copied, 0.000631599 s, 158 kB/s 2026-10-12 03:41:18,609 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap5ae85f50-1e_ctest-random-42862084.pcap | grep -c length 2026-10-12 03:41:18,622 - DEBUG - STDOUT: 1 2026-10-12 03:41:18,623 - DEBUG - STDERR: reading from file /tmp/tap5ae85f50-1e_ctest-random-42862084.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-10-12 03:41:18,623 - INFO - 1 packets are found in tcpdump output as expected 2026-10-12 03:41:18,623 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-10-12 03:41:20,667 - INFO - Tcpdump verification on VM 289a:736f:6cd9:4eba:c424:eeb7:0:6 passed 2026-10-12 03:41:20,892 - DEBUG - Forward and reverse flows for match {'sip': '289a:736f:6cd9:4eba:c424:eeb7:0:3', 'dip': '2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def', 'sport': '20202', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0 2026-10-12 03:41:20,893 - DEBUG - Flow verification PASSED on node: 10.0.0.189 for VMs - Sender: 289a:736f:6cd9:4eba:c424:eeb7:0:3, Receiver: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, Flow count expected: 0 0, got: 0 0 2026-10-12 03:41:20,893 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-10-12 03:41:22,900 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-10-12 03:41:25,189 - DEBUG - Forward and reverse flows for match {'sip': '289a:736f:6cd9:4eba:c424:eeb7:0:3', 'dip': '2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def', 'sport': '20202', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0 2026-10-12 03:41:25,189 - DEBUG - Flow verification PASSED on node: 10.0.0.188 for VMs - Sender: 289a:736f:6cd9:4eba:c424:eeb7:0:3, Receiver: 2a81:48ad:d784:4cb7:2b2b:34e3:8a8e:def, Flow count expected: 0 0, got: 0 0 2026-10-12 03:41:25,189 - INFO - Traffic verification for ECMP passed 2026-10-12 03:41:25,189 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-15220833-97155360 2026-10-12 03:41:25,319 - INFO - Removed intf route table 8ed0ff0e-b402-4fba-81a5-ee7fd1d955c3 from port aaa5fa9d-7143-4621-8409-4312ee4b820c 2026-10-12 03:41:25,347 - INFO - Deleted Interface route table 8ed0ff0e-b402-4fba-81a5-ee7fd1d955c3 2026-10-12 03:41:25,347 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-15220833-06021352 2026-10-12 03:41:25,483 - INFO - Removed intf route table 1ad8f91a-85d0-4001-a9da-e5eb41543da8 from port 5ae85f50-1e78-40fc-9e21-1d0dbafabcea 2026-10-12 03:41:25,512 - INFO - Deleted Interface route table 1ad8f91a-85d0-4001-a9da-e5eb41543da8 2026-10-12 03:41:25,512 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-15220833-99642127 2026-10-12 03:41:25,584 - INFO - Deleting VN ctest-vn-24776020 2026-10-12 03:41:25,621 - DEBUG - VN 3b292775-bb82-4c08-a9b4-10c31d313474 still in use: Unable to complete operation on network 3b292775-bb82-4c08-a9b4-10c31d313474. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-c8d3b4ef-cd6a-4e5d-b875-631530ec3735'] 2026-10-12 03:41:25,621 - WARNING - Deleting VN ctest-vn-24776020 failed..Will retry 2026-10-12 03:41:27,684 - DEBUG - VN 3b292775-bb82-4c08-a9b4-10c31d313474 still in use: Unable to complete operation on network 3b292775-bb82-4c08-a9b4-10c31d313474. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-bf02f30d-c0b4-4878-ab50-fed05af889cf'] 2026-10-12 03:41:27,685 - WARNING - Deleting VN ctest-vn-24776020 failed..Will retry 2026-10-12 03:41:29,820 - DEBUG - Response for deleting network () 2026-10-12 03:41:29,820 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/865b5a03-117e-439c-8f1e-f5a5de1eeeec 2026-10-12 03:41:29,825 - DEBUG - Response Code: 404 2026-10-12 03:41:29,825 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2026-10-12 03:41:29,830 - DEBUG - Requesting: http://10.0.0.38:8082/domain/e7595c33-d9f2-4727-95af-2f5d18bc40e3 2026-10-12 03:41:29,900 - DEBUG - Requesting: http://10.0.0.38:8082/project/ad58e7e4-b820-4463-867b-f3d75362f5c1 2026-10-12 03:41:29,987 - INFO - Validated that VN ctest-vn-24776020 is not found in API Server 2026-10-12 03:41:29,997 - DEBUG - VN ctest-vn-24776020 is not present in Agent 10.0.0.188 2026-10-12 03:41:30,005 - DEBUG - VN ctest-vn-24776020 is not present in Agent 10.0.0.189 2026-10-12 03:41:30,006 - INFO - Validated that VN ctest-vn-24776020 is not in any agent 2026-10-12 03:41:30,035 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-24776020 info 2026-10-12 03:41:31,314 - DEBUG - No XMPP flaps were noticed during the test
2026-10-12 03:41:31,314 - INFO - END TEST : test_ecmp_with_static_routes : PASSED[0:04:38]
2026-10-12 03:41:31,315 - INFO - -------------------------------------------------------------------------------- 2026-10-12 03:41:32,094 - INFO - Deleted project: ctest-DisablePolicyEcmpIpv6-15220833, ID : ad58e7e4-b820-4463-867b-f3d75362f5c1