2025-12-04 03:08:00,050 - INFO - Domain Default found not creating 2025-12-04 03:08:00,235 - INFO - Project ctest-TestQos-39654419 not found, creating it 2025-12-04 03:08:00,889 - INFO - Created Project:ctest-TestQos-39654419, ID : fcb8a086-405c-46bd-9594-59daac9eeb26 2025-12-04 03:08:03,421 - DEBUG - Response for create_network : {'network': {'id': '8739bdd0-5dcd-4195-b028-dcb9140463a9', 'name': 'ctest-vn-61005008', 'tenant_id': 'fcb8a086405c46bd959459daac9eeb26', 'project_id': 'fcb8a086405c46bd959459daac9eeb26', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestQos-39654419', 'ctest-vn-61005008'], 'port_security_enabled': True, 'description': ''}} 2025-12-04 03:08:03,710 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e70630f2-7838-4e53-b47f-95feaaece299', 'name': '', 'tenant_id': 'fcb8a086405c46bd959459daac9eeb26', 'network_id': '8739bdd0-5dcd-4195-b028-dcb9140463a9', 'ip_version': 4, 'cidr': '151.16.25.64/26', 'allocation_pools': [{'start': '151.16.25.66', 'end': '151.16.25.126'}], 'gateway_ip': '151.16.25.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '151.16.25.66', 'tags': [], 'project_id': 'fcb8a086405c46bd959459daac9eeb26'}} 2025-12-04 03:08:03,953 - DEBUG - Response for create_subnet : {'subnet': {'id': '21d92cad-883b-4af2-87ff-50dc8280885f', 'name': '', 'tenant_id': 'fcb8a086405c46bd959459daac9eeb26', 'network_id': '8739bdd0-5dcd-4195-b028-dcb9140463a9', 'ip_version': 6, 'cidr': '224d:ef08:17a4:9a83:cdc9:20bc::/96', 'allocation_pools': [{'start': '224d:ef08:17a4:9a83:cdc9:20bc:0:2', 'end': '224d:ef08:17a4:9a83:cdc9:20bc:ffff:fffe'}], 'gateway_ip': '224d:ef08:17a4:9a83:cdc9:20bc:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '224d:ef08:17a4:9a83:cdc9:20bc:0:2', 'tags': [], 'project_id': 'fcb8a086405c46bd959459daac9eeb26'}} 2025-12-04 03:08:03,978 - INFO - Created VN ctest-vn-61005008 2025-12-04 03:08:03,989 - DEBUG - VN ctest-vn-61005008 UUID is 8739bdd0-5dcd-4195-b028-dcb9140463a9 2025-12-04 03:08:04,406 - DEBUG - Response for create_network : {'network': {'id': '7fdd5e0a-3a99-4a64-9415-4d7ec67df4e0', 'name': 'ctest-vn-36730542', 'tenant_id': 'fcb8a086405c46bd959459daac9eeb26', 'project_id': 'fcb8a086405c46bd959459daac9eeb26', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestQos-39654419', 'ctest-vn-36730542'], 'port_security_enabled': True, 'description': ''}} 2025-12-04 03:08:04,654 - DEBUG - Response for create_subnet : {'subnet': {'id': '14193a42-8f20-4631-9ee6-dc1cf39aa6e1', 'name': '', 'tenant_id': 'fcb8a086405c46bd959459daac9eeb26', 'network_id': '7fdd5e0a-3a99-4a64-9415-4d7ec67df4e0', 'ip_version': 4, 'cidr': '199.83.196.64/26', 'allocation_pools': [{'start': '199.83.196.66', 'end': '199.83.196.126'}], 'gateway_ip': '199.83.196.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '199.83.196.66', 'tags': [], 'project_id': 'fcb8a086405c46bd959459daac9eeb26'}} 2025-12-04 03:08:04,849 - DEBUG - Response for create_subnet : {'subnet': {'id': '783a5c16-5a6d-4925-8eef-e4f56ccc0dcd', 'name': '', 'tenant_id': 'fcb8a086405c46bd959459daac9eeb26', 'network_id': '7fdd5e0a-3a99-4a64-9415-4d7ec67df4e0', 'ip_version': 6, 'cidr': '247a:931:27b:3236:f661:e419::/96', 'allocation_pools': [{'start': '247a:931:27b:3236:f661:e419:0:2', 'end': '247a:931:27b:3236:f661:e419:ffff:fffe'}], 'gateway_ip': '247a:931:27b:3236:f661:e419:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '247a:931:27b:3236:f661:e419:0:2', 'tags': [], 'project_id': 'fcb8a086405c46bd959459daac9eeb26'}} 2025-12-04 03:08:04,880 - INFO - Created VN ctest-vn-36730542 2025-12-04 03:08:04,897 - DEBUG - VN ctest-vn-36730542 UUID is 7fdd5e0a-3a99-4a64-9415-4d7ec67df4e0 2025-12-04 03:08:05,072 - DEBUG - Services list from nova: [, , , ] 2025-12-04 03:08:06,183 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4840-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4840-1) 2025-12-04 03:08:07,622 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4840-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4840-2) 2025-12-04 03:08:09,655 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4840-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4840-2) 2025-12-04 03:08:09,655 - INFO - Waiting for VM ctest-TestQos-39654419-23007708 to be up.. 2025-12-04 03:08:09,793 - DEBUG - VM is in ACTIVE state now 2025-12-04 03:08:09,793 - INFO - VM name : ctest-TestQos-39654419-23007708 2025-12-04 03:08:09,910 - DEBUG - VM ctest-TestQos-39654419-23007708 ID is d2b2b2b7-30bf-4454-9376-084f6d62b9cc 2025-12-04 03:08:09,981 - DEBUG - VM ctest-TestQos-39654419-23007708 launched on Node an-jenkins-deploy-platform-ansible-os-4840-1 2025-12-04 03:08:10,105 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/d2b2b2b7-30bf-4454-9376-084f6d62b9cc 2025-12-04 03:08:10,707 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/d2b2b2b7-30bf-4454-9376-084f6d62b9cc 2025-12-04 03:08:10,762 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/60bf7ff2-d5ad-4afc-8962-4200bc470036 2025-12-04 03:08:13,956 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-12-04 03:08:13,957 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:13,973 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:13,974 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:18,051 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-12-04 03:08:18,051 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:18,067 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:18,067 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:22,150 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-12-04 03:08:22,150 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:22,167 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:22,167 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:26,242 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-12-04 03:08:26,243 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:26,259 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:26,259 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:30,339 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-12-04 03:08:30,339 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:30,357 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:30,357 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:34,437 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-12-04 03:08:34,437 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:34,452 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:34,452 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:38,531 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-12-04 03:08:38,532 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:38,547 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:38,547 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:42,627 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-12-04 03:08:42,627 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:42,642 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:42,642 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:46,726 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-12-04 03:08:46,726 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:46,745 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:46,745 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:50,823 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-12-04 03:08:50,824 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 failed! 2025-12-04 03:08:50,838 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 151.16.25.65 and allocation pool is NOT set 2025-12-04 03:08:50,838 - DEBUG - Gateway for vn default-domain:ctest-TestQos-39654419:ctest-vn-61005008 is 224d:ef08:17a4:9a83:cdc9:20bc:0:1 and allocation pool is NOT set 2025-12-04 03:08:52,912 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.07 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=5.74 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 5.735/5.900/6.065/0.165 ms') 2025-12-04 03:08:52,912 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-23007708 passed 2025-12-04 03:08:52,984 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:08:52,985 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-23007708, IP 151.16.25.67, Port 22 2025-12-04 03:08:53,050 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:08:53,140 - DEBUG - VM ctest-TestQos-39654419-23007708 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:08:58,140 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:08:58,140 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-23007708, IP 151.16.25.67, Port 22 2025-12-04 03:08:58,196 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:08:58,282 - DEBUG - VM ctest-TestQos-39654419-23007708 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:09:03,283 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:03,283 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-23007708, IP 151.16.25.67, Port 22 2025-12-04 03:09:03,351 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:09:03,443 - DEBUG - VM ctest-TestQos-39654419-23007708 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:09:08,444 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:08,444 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-23007708, IP 151.16.25.67, Port 22 2025-12-04 03:09:08,510 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:09:08,598 - DEBUG - VM ctest-TestQos-39654419-23007708 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:09:13,599 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:13,599 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-23007708, IP 151.16.25.67, Port 22 2025-12-04 03:09:13,766 - DEBUG - VM ctest-TestQos-39654419-23007708 is ready for SSH connections 2025-12-04 03:09:13,767 - INFO - Waiting for VM ctest-TestQos-39654419-25161781 to be up.. 2025-12-04 03:09:13,865 - DEBUG - VM is in ACTIVE state now 2025-12-04 03:09:13,865 - INFO - VM name : ctest-TestQos-39654419-25161781 2025-12-04 03:09:13,949 - DEBUG - VM ctest-TestQos-39654419-25161781 ID is 771d9225-af6d-4b5a-9710-54062244eea7 2025-12-04 03:09:13,949 - DEBUG - VM ctest-TestQos-39654419-25161781 launched on Node an-jenkins-deploy-platform-ansible-os-4840-2 2025-12-04 03:09:14,044 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/771d9225-af6d-4b5a-9710-54062244eea7 2025-12-04 03:09:14,056 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/7b9ccf85-c4fa-4d4f-88b1-7c6f31ea521a 2025-12-04 03:09:15,231 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=17.7 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=6.99 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 6.986/12.321/17.656/5.335 ms') 2025-12-04 03:09:15,232 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-39654419-25161781 passed 2025-12-04 03:09:15,303 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:15,303 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-25161781, IP 151.16.25.68, Port 22 2025-12-04 03:09:15,370 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:09:15,473 - DEBUG - VM ctest-TestQos-39654419-25161781 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:09:20,474 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:20,474 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-25161781, IP 151.16.25.68, Port 22 2025-12-04 03:09:20,542 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:09:20,692 - DEBUG - VM ctest-TestQos-39654419-25161781 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:09:25,693 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:25,693 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-25161781, IP 151.16.25.68, Port 22 2025-12-04 03:09:25,890 - DEBUG - VM ctest-TestQos-39654419-25161781 is ready for SSH connections 2025-12-04 03:09:25,891 - INFO - Waiting for VM ctest-TestQos-39654419-77848257 to be up.. 2025-12-04 03:09:25,995 - DEBUG - VM is in ACTIVE state now 2025-12-04 03:09:25,995 - INFO - VM name : ctest-TestQos-39654419-77848257 2025-12-04 03:09:26,092 - DEBUG - VM ctest-TestQos-39654419-77848257 ID is 2e88ae49-4455-4913-8499-7e74bd66f5e3 2025-12-04 03:09:26,092 - DEBUG - VM ctest-TestQos-39654419-77848257 launched on Node an-jenkins-deploy-platform-ansible-os-4840-2 2025-12-04 03:09:26,185 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/2e88ae49-4455-4913-8499-7e74bd66f5e3 2025-12-04 03:09:26,197 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/609917d0-443e-4b77-b9d5-09f19cbb2b8c 2025-12-04 03:09:27,370 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=2.87 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=1.04 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.036/1.954/2.873/0.918 ms') 2025-12-04 03:09:27,370 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestQos-39654419-77848257 passed 2025-12-04 03:09:27,443 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:27,443 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-77848257, IP 199.83.196.67, Port 22 2025-12-04 03:09:27,511 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-04 03:09:27,603 - DEBUG - VM ctest-TestQos-39654419-77848257 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-04 03:09:32,604 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:32,604 - DEBUG - Waiting to SSH to VM ctest-TestQos-39654419-77848257, IP 199.83.196.67, Port 22 2025-12-04 03:09:32,775 - DEBUG - VM ctest-TestQos-39654419-77848257 is ready for SSH connections 2025-12-04 03:09:32,779 - INFO - ================================================================================ 2025-12-04 03:09:32,780 - INFO - STARTING TEST : test_qos_remark_dscp_on_vmi 2025-12-04 03:09:32,780 - INFO - TEST DESCRIPTION : Create a qos config for remarking DSCP 1 to 10 Have VMs A, B Apply the qos config to VM A Validate that packets from A to B have DSCP marked correctly 2025-12-04 03:09:34,092 - DEBUG - Nothing to compare xmpp stats {'10.0.0.22': {'10.20.0.209': '0', '10.20.0.254': '0'}, '10.0.0.130': {'10.20.0.192': '0', '10.20.0.209': '0'}} with 2025-12-04 03:09:34,093 - INFO - Initial checks done. Running the testcase now 2025-12-04 03:09:34,093 - INFO - 2025-12-04 03:09:34,099 - DEBUG - FC Dict is {'fc_id': 0, 'dscp': 10, 'dot1p': 1, 'exp': 1, 'connections': } 2025-12-04 03:09:34,361 - INFO - Created FC ['default-global-system-config', 'default-global-qos-config', 'ctest-fc-43230059'], UUID b472a6e2-646f-47b3-843c-136d8307b7f4 2025-12-04 03:09:34,713 - INFO - Created QosConfig ['default-domain', 'ctest-TestQos-39654419', 'ctest-qos_config-53634784'], UUID: b45df1bd-1078-4587-8f16-5bb9725e327b 2025-12-04 03:09:34,761 - INFO - Applying qos-config on VM 60bf7ff2-d5ad-4afc-8962-4200bc470036 2025-12-04 03:09:35,033 - DEBUG - VM is in ACTIVE state now 2025-12-04 03:09:35,285 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/d2b2b2b7-30bf-4454-9376-084f6d62b9cc 2025-12-04 03:09:35,293 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/60bf7ff2-d5ad-4afc-8962-4200bc470036 2025-12-04 03:09:35,303 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/0ad7b301-d36a-4dcf-8fef-9d04d56d4921 2025-12-04 03:09:35,314 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/4aeb6f80-bbfb-4194-b80f-a7c6168cec75 2025-12-04 03:09:35,524 - DEBUG - VM is in ACTIVE state now 2025-12-04 03:09:35,772 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/771d9225-af6d-4b5a-9710-54062244eea7 2025-12-04 03:09:35,781 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/7b9ccf85-c4fa-4d4f-88b1-7c6f31ea521a 2025-12-04 03:09:35,792 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/00107411-7132-45c5-a264-9916c6f018a2 2025-12-04 03:09:35,803 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/6a13f963-25ae-4921-8794-15deb4d451fe 2025-12-04 03:09:35,835 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:35,835 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:35,835 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-04 03:09:35,835 - INFO - Starting hping3 on ctest-TestQos-39654419-23007708, args: --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 2025-12-04 03:09:35,836 - DEBUG - Hping3 cmd : hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 151.16.25.68 1>/tmp/hping_ctest-random-21668265.log 2>/tmp/hping_ctest-random-21668265.result 2025-12-04 03:09:35,836 - DEBUG - Running remote_cmd, Cmd : hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 151.16.25.68 1>/tmp/hping_ctest-random-21668265.log 2>/tmp/hping_ctest-random-21668265.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.22, gateway password: c0ntrail123 2025-12-04 03:09:35,836 - DEBUG - nohup hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 151.16.25.68 1>/tmp/hping_ctest-random-21668265.log 2>/tmp/hping_ctest-random-21668265.result & echo $! > /tmp/hping_ctest-random-21668265.pid 2025-12-04 03:10:22,905 - DEBUG - None 2025-12-04 03:10:28,117 - DEBUG - Forward flow: {'index': '451404', 'rflow': '302480', 'sip': '151.16.25.67', 'sport': '10000', 'dip': '151.16.25.68', 'dport': '20000', 'proto': '17', 'vrf_id': '2', 'action': 'FORWARD', 'flags': ' ACTIVE | RFLOW_VALID ', 'd_vrf_id': '0', 'bytes': '252', 'pkts': '6', 'insight': '0', 'nhid': '30', 'underlay_udp_sport': '52810', 'ttl': '0', 'qos_id': '0', 'gen_id': '1', 'tcp_seq': '0', 'oflow_bytes': '0', 'oflow_packets': '0', 'underlay_gw_index': '-1'} 2025-12-04 03:10:28,119 - DEBUG - Reverse flow: {'index': '302480', 'rflow': '451404', 'sip': '151.16.25.68', 'sport': '20000', 'dip': '151.16.25.67', 'dport': '10000', 'proto': '17', 'vrf_id': '2', 'action': 'FORWARD', 'flags': ' ACTIVE | RFLOW_VALID ', 'd_vrf_id': '0', 'bytes': '350', 'pkts': '5', 'insight': '0', 'nhid': '30', 'underlay_udp_sport': '59484', 'ttl': '0', 'qos_id': '0', 'gen_id': '1', 'tcp_seq': '0', 'oflow_bytes': '0', 'oflow_packets': '0', 'underlay_gw_index': '-1'} 2025-12-04 03:10:28,119 - DEBUG - The filter pattern is ['udp', 'and', 'src port 52810'] 2025-12-04 03:10:28,119 - DEBUG - The filter string is '(udp and src port 52810)' 2025-12-04 03:10:28,208 - DEBUG - Executing command: sudo tcpdump -nni ens6 -U -vvxx '(udp and src port 52810)' -w /tmp/ens6_ctest-random-74888045.pcap 2025-12-04 03:10:33,299 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2025-12-04 03:10:35,302 - DEBUG - Ensuring hping3 instance with result file /tmp/hping_ctest-random-21668265.result on ctest-TestQos-39654419-23007708 is stopped 2025-12-04 03:10:35,303 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-21668265.pid | xargs kill , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.22, gateway password: c0ntrail123 2025-12-04 03:10:35,303 - DEBUG - cat /tmp/hping_ctest-random-21668265.pid | xargs kill 2025-12-04 03:10:36,138 - DEBUG - None 2025-12-04 03:10:36,138 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-21668265.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.22, gateway password: c0ntrail123 2025-12-04 03:10:36,138 - DEBUG - cat /tmp/hping_ctest-random-21668265.result 2025-12-04 03:10:36,489 - DEBUG - --- 151.16.25.68 hping statistic --- 14 packets transmitted, 14 packets received, 0% packet loss round-trip min/avg/max = 14.4/6510.5/13014.8 ms 2025-12-04 03:10:36,489 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-21668265.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.22, gateway password: c0ntrail123 2025-12-04 03:10:36,489 - DEBUG - cat /tmp/hping_ctest-random-21668265.log 2025-12-04 03:10:36,839 - DEBUG - HPING 151.16.25.68 (eth0 151.16.25.68): udp mode set, 28 headers + 0 data bytes ICMP Port Unreachable from ip=151.16.25.68 status=0 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=151.16.25.68 status=1 port=10000 seq=0 2025-12-04 03:10:36,840 - DEBUG - Hping3 stats: {'sent': '14', 'received': '14', 'loss': '0', 'rtt_min': '14.4', 'rtt_avg': '6510.5', 'rtt_max': '13014.8'} 2025-12-04 03:10:36,840 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-74888045.pcap | wc -l 2025-12-04 03:10:36,856 - DEBUG - STDOUT: 16 2025-12-04 03:10:36,856 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-74888045.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2025-12-04 03:10:36,856 - DEBUG - 16 packets are found in tcpdump output file /tmp/ens6_ctest-random-74888045.pcap but expected 1, which is fine 2025-12-04 03:10:36,856 - INFO - 16 packets are found in tcpdump output as expected 2025-12-04 03:10:36,856 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2025-12-04 03:10:38,923 - DEBUG - ['/contrail-test/ens6_ctest-random-74888045.pcap'] 2025-12-04 03:10:38,924 - DEBUG - Verifying for packet number 0 2025-12-04 03:10:38,924 - DEBUG - Validated DSCP marking of 10 2025-12-04 03:10:38,924 - DEBUG - Verifying for packet number 1 2025-12-04 03:10:38,924 - DEBUG - Validated DSCP marking of 10 2025-12-04 03:10:38,924 - DEBUG - Verifying for packet number 2 2025-12-04 03:10:38,924 - DEBUG - Validated DSCP marking of 10 2025-12-04 03:10:38,924 - DEBUG - Verifying for packet number 3 2025-12-04 03:10:38,924 - DEBUG - Validated DSCP marking of 10 2025-12-04 03:10:38,924 - INFO - Packet QoS marking validation passed 2025-12-04 03:10:38,924 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-74888045.pcap | wc -l 2025-12-04 03:10:38,938 - DEBUG - STDOUT: 16 2025-12-04 03:10:38,938 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-74888045.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2025-12-04 03:10:38,938 - DEBUG - 16 packets are found in tcpdump output file /tmp/ens6_ctest-random-74888045.pcap but expected 1, which is fine 2025-12-04 03:10:38,938 - INFO - 16 packets are found in tcpdump output as expected 2025-12-04 03:10:38,938 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2025-12-04 03:10:40,999 - DEBUG - ['/contrail-test/ens6_ctest-random-74888045.pcap'] 2025-12-04 03:10:40,999 - DEBUG - Verifying for packet number 0 2025-12-04 03:10:40,999 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-04 03:10:40,999 - DEBUG - Verifying for packet number 1 2025-12-04 03:10:40,999 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-04 03:10:40,999 - DEBUG - Verifying for packet number 2 2025-12-04 03:10:40,999 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-04 03:10:40,999 - DEBUG - Verifying for packet number 3 2025-12-04 03:10:40,999 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-04 03:10:40,999 - INFO - Packet QoS marking validation passed 2025-12-04 03:10:41,107 - INFO - Removing qos-config on VM 60bf7ff2-d5ad-4afc-8962-4200bc470036 2025-12-04 03:10:41,177 - INFO - Deleting Qos config ['default-domain', 'ctest-TestQos-39654419', 'ctest-qos_config-53634784'], UUID: b45df1bd-1078-4587-8f16-5bb9725e327b 2025-12-04 03:10:41,265 - INFO - Deleting FC ['default-global-system-config', 'default-global-qos-config', 'ctest-fc-43230059'], UUID: b472a6e2-646f-47b3-843c-136d8307b7f4 2025-12-04 03:10:42,576 - DEBUG - No XMPP flaps were noticed during the test 2025-12-04 03:10:42,576 - INFO - END TEST : test_qos_remark_dscp_on_vmi : PASSED[0:01:10] 2025-12-04 03:10:42,576 - INFO - -------------------------------------------------------------------------------- 2025-12-04 03:10:42,581 - INFO - Deleting VM ctest-TestQos-39654419-77848257 2025-12-04 03:10:42,677 - INFO - Deleting VM ctest-TestQos-39654419-25161781 2025-12-04 03:10:42,757 - INFO - Deleting VM ctest-TestQos-39654419-23007708 2025-12-04 03:10:42,846 - INFO - Deleting VN ctest-vn-36730542 2025-12-04 03:10:42,897 - DEBUG - VN 7fdd5e0a-3a99-4a64-9415-4d7ec67df4e0 still in use: Unable to complete operation on network 7fdd5e0a-3a99-4a64-9415-4d7ec67df4e0. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-f67dbf2b-578c-40cb-a666-64c487d240de'] 2025-12-04 03:10:42,898 - WARNING - Deleting VN ctest-vn-36730542 failed..Will retry 2025-12-04 03:10:45,149 - DEBUG - Response for deleting network () 2025-12-04 03:10:45,150 - INFO - Deleting VN ctest-vn-61005008 2025-12-04 03:10:45,417 - DEBUG - Response for deleting network () 2025-12-04 03:10:46,285 - INFO - Deleted project: ctest-TestQos-39654419, ID : fcb8a086-405c-46bd-9594-59daac9eeb26