is in ACTIVE state now
2026-03-09 03:14:33,694 - INFO - VM name : ctest-DisablePolicyEcmpIpv6-75228332-52141940
2026-03-09 03:14:33,778 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-75228332-52141940 ID is 8a8aa1af-2890-478f-a57e-d50de7a3fe2b
2026-03-09 03:14:33,778 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-75228332-52141940 launched on Node an-jenkins-deploy-platform-ansible-os-5454-1
2026-03-09 03:14:33,857 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/8a8aa1af-2890-478f-a57e-d50de7a3fe2b
2026-03-09 03:14:33,865 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/56f3ab8a-2ce9-45af-b9e0-81aac6f744ed
2026-03-09 03:14:35,013 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=4.00 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.407 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.407/2.201/3.996/1.794 ms')
2026-03-09 03:14:35,014 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-DisablePolicyEcmpIpv6-75228332-52141940 passed
2026-03-09 03:14:35,091 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-03-09 03:14:35,091 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-75228332-52141940, IP 2fe8:44da:fc9c:47d3:b2b9:ba87:0:6, Port 22
2026-03-09 03:14:35,253 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-75228332-52141940 is ready for SSH connections
2026-03-09 03:14:35,340 - INFO - Policy DISABLED on VMI 56f3ab8a-2ce9-45af-b9e0-81aac6f744ed
2026-03-09 03:14:35,340 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-03-09 03:14:35,340 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep 2fe8:44da:fc9c:47d3:b2b9:ba87:0:6 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2026-03-09 03:14:35,340 - DEBUG - ifconfig -a| grep 2fe8:44da:fc9c:47d3:b2b9:ba87:0:6 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1
2026-03-09 03:15:13,833 - DEBUG - eth0
2026-03-09 03:15:13,833 - DEBUG - Running remote_cmd, Cmd : sysctl net.ipv6.conf.eth0.accept_dad=0, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2026-03-09 03:15:13,833 - DEBUG - sysctl net.ipv6.conf.eth0.accept_dad=0
2026-03-09 03:15:14,324 - DEBUG - net.ipv6.conf.eth0.accept_dad = 0
2026-03-09 03:15:14,532 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-DisablePolicyEcmpIpv6-75228332', 'ctest-intf-rtb-96866895'](UUID 97549dab-6b5d-45d4-873a-a0f7f136804c), prefixes : ['26f5:c1dc:42ad:ef99:9448:b675::/96']
2026-03-09 03:15:14,600 - INFO - Added intf route table 97549dab-6b5d-45d4-873a-a0f7f136804c to port 56f3ab8a-2ce9-45af-b9e0-81aac6f744ed
2026-03-09 03:15:14,600 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2026-03-09 03:15:14,600 - DEBUG - ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1
2026-03-09 03:15:15,031 - DEBUG - eth0
2026-03-09 03:15:15,031 - DEBUG - Running remote_cmd, Cmd : ifconfig eth0 inet6 add 26f5:c1dc:42ad:ef99:9448:b675:512f:b345, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2026-03-09 03:15:15,031 - DEBUG - ifconfig eth0 inet6 add 26f5:c1dc:42ad:ef99:9448:b675:512f:b345
2026-03-09 03:15:15,519 - DEBUG - None
2026-03-09 03:15:15,519 - DEBUG - Running remote_cmd, Cmd : ifconfig -a, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2026-03-09 03:15:15,519 - DEBUG - ifconfig -a
2026-03-09 03:15:16,033 - DEBUG - eth0 Link encap:Ethernet HWaddr 02:56:f3:ab:8a:2c
inet addr:5.143.133.70 Bcast:5.143.133.127 Mask:255.255.255.192
inet6 addr: 26f5:c1dc:42ad:ef99:9448:b675:512f:b345/128 Scope:Global
inet6 addr: 2fe8:44da:fc9c:47d3:b2b9:ba87:0:6/64 Scope:Global
inet6 addr: fe80::56:f3ff:feab:8a2c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:568 errors:0 dropped:0 overruns:0 frame:0
TX packets:511 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:51423 (51.4 KB) TX bytes:43840 (43.8 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
2026-03-09 03:15:16,033 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-75228332-37666645
2026-03-09 03:15:16,172 - INFO - Removed intf route table ded906af-0eeb-4da0-876f-3d9be7fe5733 from port 025b0168-4ea8-45b4-8041-21f619f54ce2
2026-03-09 03:15:16,206 - INFO - Deleted Interface route table ded906af-0eeb-4da0-876f-3d9be7fe5733
2026-03-09 03:15:16,220 - INFO - ECMP routes found in agent 10.0.0.27, for prefix 26f5:c1dc:42ad:ef99:9448:b675::/96
2026-03-09 03:15:16,220 - INFO - Tap interface tap0f495747-6e found in ecmp routes in agent 10.0.0.27
2026-03-09 03:15:16,235 - INFO - ECMP routes found in agent 10.0.0.41, for prefix 26f5:c1dc:42ad:ef99:9448:b675::/96
2026-03-09 03:15:16,235 - INFO - Tap interface tap56f3ab8a-2c found in ecmp routes in agent 10.0.0.41
2026-03-09 03:15:16,324 - DEBUG - Executing command: sudo tcpdump -nni tap0f495747-6e -U '(udp and src host 2fe8:44da:fc9c:47d3:b2b9:ba87:0:3 and dst host 26f5:c1dc:42ad:ef99:9448:b675:512f:b345 and dst port 11000)' -w /tmp/tap0f495747-6e_ctest-random-93931919.pcap
2026-03-09 03:15:16,807 - DEBUG - Executing command: sudo tcpdump -nni tap56f3ab8a-2c -U '(udp and src host 2fe8:44da:fc9c:47d3:b2b9:ba87:0:3 and dst host 26f5:c1dc:42ad:ef99:9448:b675:512f:b345 and dst port 11000)' -w /tmp/tap56f3ab8a-2c_ctest-random-27144265.pcap
2026-03-09 03:15:17,182 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 34127 26f5:c1dc:42ad:ef99:9448:b675:512f:b345 11000 < icmp_error, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2026-03-09 03:15:17,182 - DEBUG - dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 34127 26f5:c1dc:42ad:ef99:9448:b675:512f:b345 11000 < icmp_error
2026-03-09 03:15:19,739 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.000985848 s, 101 kB/s
2026-03-09 03:15:19,739 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:19,753 - DEBUG - STDOUT: 0
2026-03-09 03:15:19,753 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:19,753 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:21,754 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:21,768 - DEBUG - STDOUT: 0
2026-03-09 03:15:21,768 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:21,768 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:23,769 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:23,783 - DEBUG - STDOUT: 0
2026-03-09 03:15:23,783 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:23,783 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:25,784 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:25,799 - DEBUG - STDOUT: 0
2026-03-09 03:15:25,799 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:25,799 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:27,800 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:27,817 - DEBUG - STDOUT: 0
2026-03-09 03:15:27,817 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:27,817 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:29,818 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:29,834 - DEBUG - STDOUT: 0
2026-03-09 03:15:29,834 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:29,835 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:31,835 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap0f495747-6e_ctest-random-93931919.pcap | grep -c length
2026-03-09 03:15:31,849 - DEBUG - STDOUT: 0
2026-03-09 03:15:31,849 - DEBUG - STDERR: reading from file /tmp/tap0f495747-6e_ctest-random-93931919.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:31,849 - WARNING - No packets are found in tcpdump output file /tmp/tap0f495747-6e_ctest-random-93931919.pcap but expected some packets
2026-03-09 03:15:31,850 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap56f3ab8a-2c_ctest-random-27144265.pcap | grep -c length
2026-03-09 03:15:31,863 - DEBUG - STDOUT: 1
2026-03-09 03:15:31,863 - DEBUG - STDERR: reading from file /tmp/tap56f3ab8a-2c_ctest-random-27144265.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-03-09 03:15:31,863 - INFO - 1 packets are found in tcpdump output as expected
2026-03-09 03:15:31,863 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}')
2026-03-09 03:15:33,914 - INFO - Tcpdump verification on VM 2fe8:44da:fc9c:47d3:b2b9:ba87:0:6 passed
2026-03-09 03:15:34,256 - DEBUG - Forward and reverse flows for match {'sip': '2fe8:44da:fc9c:47d3:b2b9:ba87:0:3', 'dip': '26f5:c1dc:42ad:ef99:9448:b675:512f:b345', 'sport': '34127', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0
2026-03-09 03:15:34,257 - DEBUG - Flow verification PASSED on node: 10.0.0.41 for VMs - Sender: 2fe8:44da:fc9c:47d3:b2b9:ba87:0:3, Receiver: 26f5:c1dc:42ad:ef99:9448:b675:512f:b345, Flow count expected: 0 0, got: 0 0
2026-03-09 03:15:34,257 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}')
2026-03-09 03:15:36,265 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}')
2026-03-09 03:15:38,564 - DEBUG - Forward and reverse flows for match {'sip': '2fe8:44da:fc9c:47d3:b2b9:ba87:0:3', 'dip': '26f5:c1dc:42ad:ef99:9448:b675:512f:b345', 'sport': '34127', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0
2026-03-09 03:15:38,565 - DEBUG - Flow verification PASSED on node: 10.0.0.41 for VMs - Sender: 2fe8:44da:fc9c:47d3:b2b9:ba87:0:3, Receiver: 26f5:c1dc:42ad:ef99:9448:b675:512f:b345, Flow count expected: 0 0, got: 0 0
2026-03-09 03:15:38,565 - INFO - Traffic verification for ECMP passed
2026-03-09 03:15:38,565 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-75228332-52141940
2026-03-09 03:15:38,704 - INFO - Removed intf route table 97549dab-6b5d-45d4-873a-a0f7f136804c from port 56f3ab8a-2ce9-45af-b9e0-81aac6f744ed
2026-03-09 03:15:38,736 - INFO - Deleted Interface route table 97549dab-6b5d-45d4-873a-a0f7f136804c
2026-03-09 03:15:38,736 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-75228332-52688325
2026-03-09 03:15:38,896 - INFO - Removed intf route table b13fc306-060c-4d5c-a98b-c72a759f98f0 from port 0f495747-6eb0-4abd-ab0a-e14368502b19
2026-03-09 03:15:38,927 - INFO - Deleted Interface route table b13fc306-060c-4d5c-a98b-c72a759f98f0
2026-03-09 03:15:38,927 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-75228332-25934600
2026-03-09 03:15:38,994 - INFO - Deleting VN ctest-vn-26530558
2026-03-09 03:15:39,046 - DEBUG - VN a1f86467-6ea2-4220-949a-cb8ee3bd370d still in use: Unable to complete operation on network a1f86467-6ea2-4220-949a-cb8ee3bd370d. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-81a10fbb-b660-4021-922d-5d795a6346e9']
2026-03-09 03:15:39,047 - WARNING - Deleting VN ctest-vn-26530558 failed..Will retry
2026-03-09 03:15:41,219 - DEBUG - Response for deleting network ()
2026-03-09 03:15:41,220 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/8fb20e17-105b-4bf9-bf85-9f185bc9d848
2026-03-09 03:15:41,225 - DEBUG - Response Code: 404
2026-03-09 03:15:41,225 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2026-03-09 03:15:41,231 - DEBUG - Requesting: http://10.0.0.22:8082/domain/65872af6-2901-4194-bbe6-c1dbc9d32dc7
2026-03-09 03:15:41,246 - DEBUG - Requesting: http://10.0.0.22:8082/project/0e8b7232-97e5-4900-bfb9-7449bdde15b6
2026-03-09 03:15:41,341 - INFO - Validated that VN ctest-vn-26530558 is not found in API Server
2026-03-09 03:15:41,351 - DEBUG - VN ctest-vn-26530558 is not present in Agent 10.0.0.41
2026-03-09 03:15:41,365 - DEBUG - VN ctest-vn-26530558 is not present in Agent 10.0.0.27
2026-03-09 03:15:41,365 - INFO - Validated that VN ctest-vn-26530558 is not in any agent
2026-03-09 03:15:41,397 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-26530558 info
2026-03-09 03:15:42,674 - DEBUG - No XMPP flaps were noticed during the test
2026-03-09 03:15:42,674 - INFO - END TEST : test_ecmp_with_static_routes : PASSED[0:04:26]
2026-03-09 03:15:42,674 - INFO - --------------------------------------------------------------------------------
2026-03-09 03:15:43,467 - INFO - Deleted project: ctest-DisablePolicyEcmpIpv6-75228332, ID : 0e8b7232-97e5-4900-bfb9-7449bdde15b6