2026-08-18 07:01:08,416 - DEBUG - [10.0.0.33]: Running cmd : hostname 2026-08-18 07:01:08,712 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-2070-1 2026-08-18 07:01:08,712 - DEBUG - [10.0.0.33]: Running cmd : hostname -f 2026-08-18 07:01:08,758 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-2070-1 2026-08-18 07:01:08,758 - DEBUG - [10.0.0.33]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-08-18 07:01:08,836 - DEBUG - Output : NAMES contrail_test_VSvptJfDu contrail-kubernetes-master-kubemanager-1 analytics_database-nodemgr-1 analytics_database-cassandra-1 analytics_database-query-engine-1 analytics_snmp-topology-1 analytics_snmp-nodemgr-1 analytics_snmp-snmp-collector-1 analytics_alarm-kafka-1 analytics_alarm-alarm-gen-1 analytics_alarm-nodemgr-1 analytics-collector-1 analytics-nodemgr-1 analytics-api-1 redis-redis-1 webui-web-1 webui-job-1 control-named-1 control-dns-1 control-control-1 control-nodemgr-1 config_database-zookeeper-1 config_database-nodemgr-1 config_database-rabbitmq-1 config_database-cassandra-1 config_api-devicemgr-1 config_api-dnsmasq-1 config_api-nodemgr-1 config_api-svcmonitor-1 config_api-schema-1 config_api-api-1 2026-08-18 07:01:08,837 - DEBUG - [10.0.0.33]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2026-08-18 07:01:08,926 - DEBUG - Output : 127.0.0.1/8 10.0.0.33/24 252.33.0.1/8 10.120.203.1/24 172.17.0.1/16 2026-08-18 07:01:08,926 - DEBUG - [10.0.0.33]: Running cmd : getent hosts 10.0.0.33 | head -n 1 | awk '{print $2}' 2026-08-18 07:01:08,972 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-2070-1 2026-08-18 07:01:09,079 - DEBUG - [10.0.0.22]: Running cmd : hostname 2026-08-18 07:01:09,330 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-2070-1 2026-08-18 07:01:09,331 - DEBUG - [10.0.0.22]: Running cmd : hostname -f 2026-08-18 07:01:09,366 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-2070-1 2026-08-18 07:01:09,366 - DEBUG - [10.0.0.22]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-08-18 07:01:09,421 - DEBUG - Output : NAMES vrouter-vrouter-agent-1 vrouter-nodemgr-1 2026-08-18 07:01:09,422 - DEBUG - [10.0.0.22]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2026-08-18 07:01:09,493 - DEBUG - Output : 10.0.0.22/24 2026-08-18 07:01:09,493 - DEBUG - [10.0.0.22]: Running cmd : getent hosts 10.0.0.22 | head -n 1 | awk '{print $2}' 2026-08-18 07:01:09,528 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-2070-1 2026-08-18 07:01:09,609 - INFO - Using existing project ['default-domain', 'k8s-default'](6f05b055-7859-4ade-9a35-8c31d0d12a4a) 2026-08-18 07:01:09,747 - INFO - Using existing project ['default-domain', 'k8s-default'](6f05b055-7859-4ade-9a35-8c31d0d12a4a) 2026-08-18 07:01:09,878 - INFO - ================================================================================ 2026-08-18 07:01:09,878 - INFO - STARTING TEST : test_ingress_ip_assignment 2026-08-18 07:01:09,878 - INFO - TEST DESCRIPTION : Verify that Ingress gets a CLuster IP which is reachable to Pods in same namespace. Also verify that a Floating IP is assigned to the Ingress from the Public FIP poo. Steps: 1. Create a service with 2 pods running nginx 2. Create an ingress out of this service 3. From another Pod do a wget on the ingress Cluster ip Validate that Ingress get a IP from Public FIP pool which might/might not be accessible. Validate that service and its loadbalancing work 2026-08-18 07:01:10,140 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2026-08-18 07:01:10,149 - DEBUG - Nothing to compare xmpp stats {'10.0.0.22': {'10.0.0.33': '0'}} with 2026-08-18 07:01:10,149 - INFO - Initial checks done. Running the testcase now 2026-08-18 07:01:10,149 - INFO - 2026-08-18 07:01:10,154 - INFO - Namespace default already exists 2026-08-18 07:01:10,155 - DEBUG - [10.0.0.33]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf' 2026-08-18 07:01:10,284 - DEBUG - Output : cluster_project={} 2026-08-18 07:01:10,340 - INFO - Project default-domain:k8s-default for Namespace default is seen in contrail-api 2026-08-18 07:01:10,340 - DEBUG - Project uuid in Contrail is 6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:01:10,340 - DEBUG - [10.0.0.33]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN' 2026-08-18 07:01:10,489 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3 2026-08-18 07:01:10,489 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NamespaceDatabaseList?namespace_uuid=5b072236-fe9f-4578-aa87-a3cba521d99c 2026-08-18 07:01:10,494 - INFO - Namespace default with uuid 5b072236-fe9f-4578-aa87-a3cba521d99c found in kube manager 2026-08-18 07:01:10,494 - INFO - Namespace default verification passed 2026-08-18 07:01:10,497 - DEBUG - Service ctest-nginx-svc-87213774 not present 2026-08-18 07:01:10,498 - INFO - Creating service ctest-nginx-svc-87213774 2026-08-18 07:01:10,529 - INFO - Created Service ctest-nginx-svc-87213774 2026-08-18 07:01:10,560 - DEBUG - Pod ctest-nginx-pod-52240711 not present 2026-08-18 07:01:10,563 - INFO - Creating Pod ctest-nginx-pod-52240711 2026-08-18 07:01:10,645 - DEBUG - Pod : ctest-nginx-pod-52240711 UUID is 8cd6c81f-66e7-4cc0-a74c-b1457edd243c 2026-08-18 07:01:10,650 - DEBUG - Pod ctest-nginx-pod-09006782 not present 2026-08-18 07:01:10,651 - INFO - Creating Pod ctest-nginx-pod-09006782 2026-08-18 07:01:10,698 - DEBUG - Pod : ctest-nginx-pod-09006782 UUID is fe13d5e8-be16-4d59-a05b-eedb4ae58137 2026-08-18 07:01:10,937 - INFO - Created VN __public__, UUID :cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:11,174 - DEBUG - Requesting: http://10.0.0.33:8082/domains 2026-08-18 07:01:11,180 - DEBUG - Requesting: http://10.0.0.33:8082/domain/92e53181-b534-4769-99af-1241477f3b38 2026-08-18 07:01:11,218 - DEBUG - Requesting: http://10.0.0.33:8082/project/6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:01:11,276 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:11,297 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:11,309 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:11,316 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:11,323 - DEBUG - Route Targets: [] 2026-08-18 07:01:11,323 - DEBUG - RT names not yet present for VN __public__ 2026-08-18 07:01:16,324 - DEBUG - Requesting: http://10.0.0.33:8082/domains 2026-08-18 07:01:16,335 - DEBUG - Requesting: http://10.0.0.33:8082/domain/92e53181-b534-4769-99af-1241477f3b38 2026-08-18 07:01:16,346 - DEBUG - Requesting: http://10.0.0.33:8082/project/6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:01:16,416 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:16,426 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:16,437 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:16,448 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:16,455 - DEBUG - Requesting: http://10.0.0.33:8082/route-target/9e3cba77-19a9-40d5-8491-df32f1c1cece 2026-08-18 07:01:16,462 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-08-18 07:01:16,462 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:16,471 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:16,485 - INFO - Verified VN network id 13 for VN cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:16,485 - INFO - Verifications in API Server for VN __public__ passed 2026-08-18 07:01:16,485 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:16,499 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:16,507 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:01:16,516 - DEBUG - Requesting: http://10.0.0.33:8082/route-target/9e3cba77-19a9-40d5-8491-df32f1c1cece 2026-08-18 07:01:16,523 - DEBUG - Requesting: http://10.0.0.33:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-08-18 07:01:16,532 - DEBUG - Control-node 10.0.0.33 : VN object is : {'node_name': 'virtual-network:default-domain:k8s-default:__public__', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'cloud-admin', 'owner-access': '7', 'group': 'cloud-admin-group', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14946512798767599250', 'uuid-lslong': '12548260203017183144'}, 'enable': 'true', 'created': '2026-08-18T07:01:10', 'last-modified': '2026-08-18T07:01:11', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.429275'} 2026-08-18 07:01:16,532 - DEBUG - Requesting: http://10.0.0.33:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:01:16,543 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-08-18 07:01:16,543 - INFO - On all control nodes, Config, RI and RT verification for VN __public__ passed 2026-08-18 07:01:16,544 - DEBUG - ====Verifying policy data for __public__ in API_Server ====== 2026-08-18 07:01:16,544 - DEBUG - Requesting: http://10.0.0.33:8082/domains 2026-08-18 07:01:16,550 - DEBUG - Requesting: http://10.0.0.33:8082/domain/92e53181-b534-4769-99af-1241477f3b38 2026-08-18 07:01:16,564 - DEBUG - Requesting: http://10.0.0.33:8082/project/6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:01:16,611 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:16,626 - DEBUG - =>VN __public__ has no policy to be verified 2026-08-18 07:01:16,626 - DEBUG - Verifying the vn in opserver 2026-08-18 07:01:16,626 - DEBUG - Verifying the default-domain:k8s-default:__public__ virtual network link through opserver 10.0.0.33 2026-08-18 07:01:16,626 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-networks 2026-08-18 07:01:16,631 - DEBUG - vn link and name as {'name': 'default-domain:k8s-default:__public__', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-network/default-domain:k8s-default:__public__?flat'} 2026-08-18 07:01:16,631 - INFO - Validated that VN default-domain:k8s-default:__public__ is found in opserver 2026-08-18 07:01:16,631 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:01:16,640 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:01:16,652 - DEBUG - Do not have enough data to verify VN in agent 2026-08-18 07:01:16,652 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:01:16,659 - DEBUG - VRF ids for VN __public__: {} 2026-08-18 07:01:16,760 - INFO - Creating Floating IP pool __fip_pool_public__ in API Server 2026-08-18 07:01:16,936 - DEBUG - Requesting: http://10.0.0.33:8082/domains 2026-08-18 07:01:16,942 - DEBUG - Requesting: http://10.0.0.33:8082/domain/92e53181-b534-4769-99af-1241477f3b38 2026-08-18 07:01:16,951 - DEBUG - Requesting: http://10.0.0.33:8082/project/6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:01:17,010 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:17,023 - DEBUG - Requesting: http://10.0.0.33:8082/floating-ip-pool/19803e9f-2898-4b82-a6bd-451298b2849e 2026-08-18 07:01:17,031 - DEBUG - Requesting: http://10.0.0.33:8082/domains 2026-08-18 07:01:17,038 - DEBUG - Requesting: http://10.0.0.33:8082/domain/92e53181-b534-4769-99af-1241477f3b38 2026-08-18 07:01:17,046 - DEBUG - Requesting: http://10.0.0.33:8082/project/6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:01:17,096 - DEBUG - Requesting: http://10.0.0.33:8082/virtual-network/cf6cae33-515f-4e92-ae24-602bce029fa8 2026-08-18 07:01:17,109 - DEBUG - FIP Pool __fip_pool_public__ found in API Server 2026-08-18 07:01:17,109 - INFO - Verification for FIP pool __fip_pool_public__ in API Server passed 2026-08-18 07:01:17,117 - DEBUG - Requesting: http://10.0.0.33:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-08-18 07:01:17,124 - DEBUG - Control-node Ifmap-view has FIP pool __fip_pool_public__ information 2026-08-18 07:01:17,124 - INFO - Verification for FIP pool __fip_pool_public__ in Control-node passed 2026-08-18 07:01:17,129 - DEBUG - Ingress ctest-nginx-ingress-41131201 not present 2026-08-18 07:01:17,130 - INFO - Creating Ingress ctest-nginx-ingress-41131201 2026-08-18 07:01:17,141 - INFO - Created Ingress ctest-nginx-ingress-41131201 2026-08-18 07:01:17,147 - DEBUG - Cluster IP not yet seen for Ingress ctest-nginx-ingress-41131201 2026-08-18 07:01:20,154 - DEBUG - For Ingress ctest-nginx-ingress-41131201, Cluster IP: 10.47.255.249, External IPs ['206.192.159.124'] 2026-08-18 07:01:20,154 - INFO - Verifications in k8s passed for Ingress ctest-nginx-ingress-41131201 2026-08-18 07:01:20,154 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_LoadbalancerDatabaseList?x=40ffb877-b33d-4f30-abb1-c0068ec51697 2026-08-18 07:01:20,163 - INFO - Ingress ctest-nginx-ingress-41131201 with uuid 40ffb877-b33d-4f30-abb1-c0068ec51697 found in kube manager 2026-08-18 07:01:20,163 - INFO - Ingress ctest-nginx-ingress-41131201 verification passed 2026-08-18 07:01:20,166 - DEBUG - Pod ctest-busybox-pod-57263240 not present 2026-08-18 07:01:20,167 - INFO - Creating Pod ctest-busybox-pod-57263240 2026-08-18 07:01:20,209 - DEBUG - Pod : ctest-busybox-pod-57263240 UUID is 685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:20,224 - DEBUG - Pod ctest-nginx-pod-52240711 not in running state.Currently in Pending 2026-08-18 07:01:25,233 - INFO - Pod ctest-nginx-pod-52240711 is in running state.Got IP 10.47.255.251 2026-08-18 07:01:25,248 - DEBUG - Pod ctest-nginx-pod-52240711 has vmi 98e6c92e-9ad2-11f1-b893-fa163e62c7f3 2026-08-18 07:01:25,248 - INFO - Verified pod ctest-nginx-pod-52240711 in contrail-api 2026-08-18 07:01:25,248 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-08-18 07:01:25,260 - DEBUG - VMI 98e6c92e-9ad2-11f1-b893-fa163e62c7f3 is active in agent 10.0.0.22 2026-08-18 07:01:25,260 - INFO - Verified Pod ctest-nginx-pod-52240711 in agent 10.0.0.22 2026-08-18 07:01:25,260 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=8cd6c81f-66e7-4cc0-a74c-b1457edd243c 2026-08-18 07:01:25,265 - INFO - Pod ctest-nginx-pod-52240711 with uuid 8cd6c81f-66e7-4cc0-a74c-b1457edd243c found in kube manager 2026-08-18 07:01:25,265 - INFO - Pod ctest-nginx-pod-52240711 verification passed 2026-08-18 07:01:25,336 - DEBUG - [Pod ctest-nginx-pod-52240711] Cmd: echo ctest-nginx-pod-52240711 > /usr/share/nginx/html/index.html, Output: 2026-08-18 07:01:25,344 - INFO - Pod ctest-nginx-pod-09006782 is in running state.Got IP 10.47.255.250 2026-08-18 07:01:25,358 - DEBUG - Pod ctest-nginx-pod-09006782 has vmi 9951c710-9ad2-11f1-b893-fa163e62c7f3 2026-08-18 07:01:25,358 - INFO - Verified pod ctest-nginx-pod-09006782 in contrail-api 2026-08-18 07:01:25,359 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-08-18 07:01:25,370 - DEBUG - VMI 9951c710-9ad2-11f1-b893-fa163e62c7f3 is active in agent 10.0.0.22 2026-08-18 07:01:25,370 - INFO - Verified Pod ctest-nginx-pod-09006782 in agent 10.0.0.22 2026-08-18 07:01:25,370 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=fe13d5e8-be16-4d59-a05b-eedb4ae58137 2026-08-18 07:01:25,375 - INFO - Pod ctest-nginx-pod-09006782 with uuid fe13d5e8-be16-4d59-a05b-eedb4ae58137 found in kube manager 2026-08-18 07:01:25,375 - INFO - Pod ctest-nginx-pod-09006782 verification passed 2026-08-18 07:01:25,441 - DEBUG - [Pod ctest-nginx-pod-09006782] Cmd: echo ctest-nginx-pod-09006782 > /usr/share/nginx/html/index.html, Output: 2026-08-18 07:01:25,449 - DEBUG - Pod ctest-busybox-pod-57263240 not in running state.Currently in Pending 2026-08-18 07:01:30,459 - INFO - Pod ctest-busybox-pod-57263240 is in running state.Got IP 10.47.255.248 2026-08-18 07:01:30,515 - DEBUG - Pod ctest-busybox-pod-57263240 has vmi 9df3a900-9ad2-11f1-b893-fa163e62c7f3 2026-08-18 07:01:30,516 - INFO - Verified pod ctest-busybox-pod-57263240 in contrail-api 2026-08-18 07:01:30,516 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-08-18 07:01:30,527 - DEBUG - VMI 9df3a900-9ad2-11f1-b893-fa163e62c7f3 is active in agent 10.0.0.22 2026-08-18 07:01:30,527 - INFO - Verified Pod ctest-busybox-pod-57263240 in agent 10.0.0.22 2026-08-18 07:01:30,527 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:30,532 - INFO - Pod ctest-busybox-pod-57263240 with uuid 685f310c-b8fe-46ce-a26c-61a65521f182 found in kube manager 2026-08-18 07:01:30,532 - INFO - Pod ctest-busybox-pod-57263240 verification passed 2026-08-18 07:01:30,602 - DEBUG - [Pod ctest-busybox-pod-57263240] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout ctest-nginx-pod-52240711 2026-08-18 07:01:30,602 - DEBUG - [Pod Pod ctest-busybox-pod-57263240] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-08-18 07:01:30,602 - DEBUG - [Pod Pod ctest-busybox-pod-57263240] Cmd output: Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout ctest-nginx-pod-52240711 2026-08-18 07:01:30,658 - DEBUG - [Pod ctest-busybox-pod-57263240] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-09006782 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-08-18 07:01:30,658 - DEBUG - [Pod Pod ctest-busybox-pod-57263240] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-08-18 07:01:30,658 - DEBUG - [Pod Pod ctest-busybox-pod-57263240] Cmd output: ctest-nginx-pod-09006782 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-08-18 07:01:30,658 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-nginx-pod-52240711': 1, 'ctest-nginx-pod-09006782': 1} 2026-08-18 07:01:30,658 - WARNING - Nothing to delete parallely 2026-08-18 07:01:30,659 - INFO - Deleting pod default:ctest-busybox-pod-57263240 2026-08-18 07:01:30,680 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:30,688 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:01:35,689 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:35,696 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:01:40,696 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:40,704 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:01:45,705 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:45,714 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:01:50,715 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:50,724 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:01:55,724 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:01:55,732 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:02:00,733 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:02:00,741 - WARNING - Pod uuid 685f310c-b8fe-46ce-a26c-61a65521f182 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:02:05,742 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=685f310c-b8fe-46ce-a26c-61a65521f182 2026-08-18 07:02:05,750 - DEBUG - Pod 685f310c-b8fe-46ce-a26c-61a65521f182 is not in agent 10.0.0.22 VM list 2026-08-18 07:02:05,750 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9df3a900-9ad2-11f1-b893-fa163e62c7f3 2026-08-18 07:02:05,757 - DEBUG - VMI 9df3a900-9ad2-11f1-b893-fa163e62c7f3 is removed from agent 10.0.0.22 2026-08-18 07:02:05,757 - INFO - Verified that pod ctest-busybox-pod-57263240 is removed in agent 2026-08-18 07:02:05,758 - INFO - Deleting Ingress : ctest-nginx-ingress-41131201 2026-08-18 07:02:05,769 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_LoadbalancerDatabaseList?x=40ffb877-b33d-4f30-abb1-c0068ec51697 2026-08-18 07:02:05,777 - ERROR - Ingress ctest-nginx-ingress-41131201 with uuid 40ffb877-b33d-4f30-abb1-c0068ec51697 still found in kube manager 2026-08-18 07:02:06,778 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_LoadbalancerDatabaseList?x=40ffb877-b33d-4f30-abb1-c0068ec51697 2026-08-18 07:02:06,783 - INFO - Ingress ctest-nginx-ingress-41131201 with uuid 40ffb877-b33d-4f30-abb1-c0068ec51697 deleted successfully from kube manager 2026-08-18 07:02:06,783 - INFO - Verifications on Ingress %s cleanup passed 2026-08-18 07:02:06,783 - INFO - Deleting the FIP pool __fip_pool_public__ 2026-08-18 07:02:07,127 - DEBUG - Requesting: http://10.0.0.33:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-08-18 07:02:07,220 - DEBUG - Control-node Ifmap-view does not have FIP pool __fip_pool_public__ information 2026-08-18 07:02:07,221 - INFO - Deleting VN __public__ 2026-08-18 07:02:07,613 - DEBUG - Requesting: http://10.0.0.33:8082/routing-instance/2b83d611-4fa1-4b06-b481-ee4b690782a4 2026-08-18 07:02:07,632 - DEBUG - Response Code: 404 2026-08-18 07:02:07,633 - DEBUG - Requesting: http://10.0.0.33:8082/domains 2026-08-18 07:02:07,660 - DEBUG - Requesting: http://10.0.0.33:8082/domain/92e53181-b534-4769-99af-1241477f3b38 2026-08-18 07:02:07,681 - DEBUG - Requesting: http://10.0.0.33:8082/project/6f05b055-7859-4ade-9a35-8c31d0d12a4a 2026-08-18 07:02:07,831 - INFO - Validated that VN __public__ is not found in API Server 2026-08-18 07:02:07,831 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VnListReq?name=default-domain:k8s-default:__public__ 2026-08-18 07:02:07,849 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:02:07,863 - DEBUG - VN __public__ is not present in Agent 10.0.0.22 2026-08-18 07:02:07,863 - INFO - Validated that VN __public__ is not in any agent 2026-08-18 07:02:07,863 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:02:07,901 - DEBUG - Requesting: http://10.0.0.33:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-08-18 07:02:07,910 - DEBUG - Requesting: http://10.0.0.33:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-08-18 07:02:07,917 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN __public__ info 2026-08-18 07:02:07,918 - INFO - Deleting pod default:ctest-nginx-pod-09006782 2026-08-18 07:02:07,949 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=fe13d5e8-be16-4d59-a05b-eedb4ae58137 2026-08-18 07:02:07,970 - WARNING - Pod uuid fe13d5e8-be16-4d59-a05b-eedb4ae58137 is still seen in agent 10.0.0.22 VM list 2026-08-18 07:02:12,970 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=fe13d5e8-be16-4d59-a05b-eedb4ae58137 2026-08-18 07:02:12,978 - DEBUG - Pod fe13d5e8-be16-4d59-a05b-eedb4ae58137 is not in agent 10.0.0.22 VM list 2026-08-18 07:02:12,978 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9951c710-9ad2-11f1-b893-fa163e62c7f3 2026-08-18 07:02:12,985 - DEBUG - VMI 9951c710-9ad2-11f1-b893-fa163e62c7f3 is removed from agent 10.0.0.22 2026-08-18 07:02:12,985 - INFO - Verified that pod ctest-nginx-pod-09006782 is removed in agent 2026-08-18 07:02:12,986 - INFO - Deleting pod default:ctest-nginx-pod-52240711 2026-08-18 07:02:13,002 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=8cd6c81f-66e7-4cc0-a74c-b1457edd243c 2026-08-18 07:02:13,010 - WARNING - Pod uuid 8cd6c81f-66e7-4cc0-a74c-b1457edd243c is still seen in agent 10.0.0.22 VM list 2026-08-18 07:02:18,010 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=8cd6c81f-66e7-4cc0-a74c-b1457edd243c 2026-08-18 07:02:18,018 - DEBUG - Pod 8cd6c81f-66e7-4cc0-a74c-b1457edd243c is not in agent 10.0.0.22 VM list 2026-08-18 07:02:18,018 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:98e6c92e-9ad2-11f1-b893-fa163e62c7f3 2026-08-18 07:02:18,025 - DEBUG - VMI 98e6c92e-9ad2-11f1-b893-fa163e62c7f3 is removed from agent 10.0.0.22 2026-08-18 07:02:18,025 - INFO - Verified that pod ctest-nginx-pod-52240711 is removed in agent 2026-08-18 07:02:18,025 - INFO - Deleting service : ctest-nginx-svc-87213774 2026-08-18 07:02:18,265 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq? 2026-08-18 07:02:18,272 - DEBUG - No XMPP flaps were noticed during the test