2026-07-30 14:34:34,249 - DEBUG - [10.0.0.177]: Running cmd : hostname 2026-07-30 14:34:34,579 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-2056-30 2026-07-30 14:34:34,579 - DEBUG - [10.0.0.177]: Running cmd : hostname -f 2026-07-30 14:34:34,635 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-2056-30 2026-07-30 14:34:34,635 - DEBUG - [10.0.0.177]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-07-30 14:34:34,763 - DEBUG - Output : NAMES contrail_test_MwDDMb8qo contrail-kubernetes-master-kubemanager-1 webui-job-1 webui-web-1 analytics_database-query-engine-1 analytics_database-cassandra-1 analytics_database-nodemgr-1 analytics_snmp-snmp-collector-1 analytics_snmp-topology-1 analytics_snmp-nodemgr-1 analytics_alarm-kafka-1 analytics_alarm-alarm-gen-1 analytics_alarm-nodemgr-1 analytics-api-1 analytics-nodemgr-1 analytics-collector-1 redis-redis-1 control-named-1 control-nodemgr-1 control-dns-1 control-control-1 config_database-rabbitmq-1 config_database-cassandra-1 config_database-nodemgr-1 config_database-zookeeper-1 config_api-devicemgr-1 config_api-nodemgr-1 config_api-svcmonitor-1 config_api-schema-1 config_api-dnsmasq-1 config_api-api-1 2026-07-30 14:34:34,763 - DEBUG - [10.0.0.177]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2026-07-30 14:34:34,817 - DEBUG - Output : 127.0.0.1/8 10.0.0.177/24 252.177.0.1/8 10.230.61.1/24 172.17.0.1/16 2026-07-30 14:34:34,818 - DEBUG - [10.0.0.177]: Running cmd : getent hosts 10.0.0.177 | head -n 1 | awk '{print $2}' 2026-07-30 14:34:34,878 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-2056-30 2026-07-30 14:34:35,007 - DEBUG - [10.0.0.38]: Running cmd : hostname 2026-07-30 14:34:35,334 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-2056-1 2026-07-30 14:34:35,334 - DEBUG - [10.0.0.38]: Running cmd : hostname -f 2026-07-30 14:34:35,380 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-2056-1 2026-07-30 14:34:35,380 - DEBUG - [10.0.0.38]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-07-30 14:34:35,457 - DEBUG - Output : NAMES vrouter-vrouter-agent-1 vrouter-nodemgr-1 2026-07-30 14:34:35,458 - DEBUG - [10.0.0.38]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2026-07-30 14:34:35,536 - DEBUG - Output : 10.0.0.38/24 2026-07-30 14:34:35,536 - DEBUG - [10.0.0.38]: Running cmd : getent hosts 10.0.0.38 | head -n 1 | awk '{print $2}' 2026-07-30 14:34:35,583 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-2056-1 2026-07-30 14:34:35,727 - INFO - Using existing project ['default-domain', 'k8s-default'](43237206-e1eb-4ffd-a906-340c6ed9da44) 2026-07-30 14:34:35,875 - INFO - Using existing project ['default-domain', 'k8s-default'](43237206-e1eb-4ffd-a906-340c6ed9da44) 2026-07-30 14:34:36,077 - INFO - ================================================================================ 2026-07-30 14:34:36,078 - INFO - STARTING TEST : test_ingress_ip_assignment 2026-07-30 14:34:36,079 - INFO - TEST DESCRIPTION : Verify that Ingress gets a CLuster IP which is reachable to Pods in same namespace. Also verify that a Floating IP is assigned to the Ingress from the Public FIP poo. Steps: 1. Create a service with 2 pods running nginx 2. Create an ingress out of this service 3. From another Pod do a wget on the ingress Cluster ip Validate that Ingress get a IP from Public FIP pool which might/might not be accessible. Validate that service and its loadbalancing work 2026-07-30 14:34:36,363 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq? 2026-07-30 14:34:36,373 - DEBUG - Nothing to compare xmpp stats {'10.0.0.38': {'10.0.0.177': '0'}} with 2026-07-30 14:34:36,373 - INFO - Initial checks done. Running the testcase now 2026-07-30 14:34:36,374 - INFO - 2026-07-30 14:34:36,378 - INFO - Namespace default already exists 2026-07-30 14:34:36,378 - DEBUG - [10.0.0.177]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf' 2026-07-30 14:34:36,627 - DEBUG - Output : cluster_project={} 2026-07-30 14:34:36,743 - INFO - Project default-domain:k8s-default for Namespace default is seen in contrail-api 2026-07-30 14:34:36,744 - DEBUG - Project uuid in Contrail is 43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:34:36,744 - DEBUG - [10.0.0.177]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN' 2026-07-30 14:34:36,983 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3 2026-07-30 14:34:36,983 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_NamespaceDatabaseList?namespace_uuid=5daf36ad-4af5-4b2d-ad9d-b051d5aa2918 2026-07-30 14:34:36,989 - INFO - Namespace default with uuid 5daf36ad-4af5-4b2d-ad9d-b051d5aa2918 found in kube manager 2026-07-30 14:34:36,989 - INFO - Namespace default verification passed 2026-07-30 14:34:36,994 - DEBUG - Service ctest-nginx-svc-08200534 not present 2026-07-30 14:34:36,995 - INFO - Creating service ctest-nginx-svc-08200534 2026-07-30 14:34:37,029 - INFO - Created Service ctest-nginx-svc-08200534 2026-07-30 14:34:37,043 - DEBUG - Pod ctest-nginx-pod-05652946 not present 2026-07-30 14:34:37,045 - INFO - Creating Pod ctest-nginx-pod-05652946 2026-07-30 14:34:37,123 - DEBUG - Pod : ctest-nginx-pod-05652946 UUID is a5e3979c-8dcc-448a-9897-4e5f6db357d1 2026-07-30 14:34:37,126 - DEBUG - Pod ctest-nginx-pod-05268493 not present 2026-07-30 14:34:37,127 - INFO - Creating Pod ctest-nginx-pod-05268493 2026-07-30 14:34:37,159 - DEBUG - Pod : ctest-nginx-pod-05268493 UUID is a9585a12-a206-472e-87b0-cd3998ea9fb5 2026-07-30 14:34:37,378 - INFO - Created VN __public__, UUID :71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:37,667 - DEBUG - Requesting: http://10.0.0.177:8082/domains 2026-07-30 14:34:37,674 - DEBUG - Requesting: http://10.0.0.177:8082/domain/dbe26e84-f679-42d9-bbe3-c4786d1d55d3 2026-07-30 14:34:37,722 - DEBUG - Requesting: http://10.0.0.177:8082/project/43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:34:37,824 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:37,877 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:37,947 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:37,978 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:38,005 - DEBUG - Route Targets: [] 2026-07-30 14:34:38,005 - DEBUG - RT names not yet present for VN __public__ 2026-07-30 14:34:43,005 - DEBUG - Requesting: http://10.0.0.177:8082/domains 2026-07-30 14:34:43,015 - DEBUG - Requesting: http://10.0.0.177:8082/domain/dbe26e84-f679-42d9-bbe3-c4786d1d55d3 2026-07-30 14:34:43,024 - DEBUG - Requesting: http://10.0.0.177:8082/project/43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:34:43,087 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,098 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,112 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:43,120 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:43,126 - DEBUG - Requesting: http://10.0.0.177:8082/route-target/3f1deb85-0d52-47b8-8020-76239e3b39b5 2026-07-30 14:34:43,132 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-07-30 14:34:43,132 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,140 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:43,160 - INFO - Verified VN network id 13 for VN 71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,160 - INFO - Verifications in API Server for VN __public__ passed 2026-07-30 14:34:43,160 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,170 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:43,181 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:34:43,190 - DEBUG - Requesting: http://10.0.0.177:8082/route-target/3f1deb85-0d52-47b8-8020-76239e3b39b5 2026-07-30 14:34:43,202 - DEBUG - Requesting: http://10.0.0.177:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-07-30 14:34:43,209 - DEBUG - Control-node 10.0.0.177 : VN object is : {'node_name': 'virtual-network:default-domain:k8s-default:__public__', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'cloud-admin', 'owner-access': '7', 'group': 'cloud-admin-group', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8197652700843623419', 'uuid-lslong': '9240978228619928747'}, 'enable': 'true', 'created': '2026-07-30T14:34:37', 'last-modified': '2026-07-30T14:34:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.650967'} 2026-07-30 14:34:43,209 - DEBUG - Requesting: http://10.0.0.177:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:34:43,222 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-07-30 14:34:43,222 - INFO - On all control nodes, Config, RI and RT verification for VN __public__ passed 2026-07-30 14:34:43,223 - DEBUG - ====Verifying policy data for __public__ in API_Server ====== 2026-07-30 14:34:43,223 - DEBUG - Requesting: http://10.0.0.177:8082/domains 2026-07-30 14:34:43,228 - DEBUG - Requesting: http://10.0.0.177:8082/domain/dbe26e84-f679-42d9-bbe3-c4786d1d55d3 2026-07-30 14:34:43,235 - DEBUG - Requesting: http://10.0.0.177:8082/project/43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:34:43,288 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,302 - DEBUG - =>VN __public__ has no policy to be verified 2026-07-30 14:34:43,302 - DEBUG - Verifying the vn in opserver 2026-07-30 14:34:43,302 - DEBUG - Verifying the default-domain:k8s-default:__public__ virtual network link through opserver 10.0.0.177 2026-07-30 14:34:43,302 - DEBUG - Requesting: http://10.0.0.177:8081/analytics/uves/virtual-networks 2026-07-30 14:34:43,314 - DEBUG - vn link and name as {'name': 'default-domain:k8s-default:__public__', 'href': 'http://10.0.0.177:8081/analytics/uves/virtual-network/default-domain:k8s-default:__public__?flat'} 2026-07-30 14:34:43,314 - INFO - Validated that VN default-domain:k8s-default:__public__ is found in opserver 2026-07-30 14:34:43,315 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:34:43,325 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:34:43,334 - DEBUG - Do not have enough data to verify VN in agent 2026-07-30 14:34:43,334 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:34:43,341 - DEBUG - VRF ids for VN __public__: {} 2026-07-30 14:34:43,478 - INFO - Creating Floating IP pool __fip_pool_public__ in API Server 2026-07-30 14:34:43,615 - DEBUG - Requesting: http://10.0.0.177:8082/domains 2026-07-30 14:34:43,621 - DEBUG - Requesting: http://10.0.0.177:8082/domain/dbe26e84-f679-42d9-bbe3-c4786d1d55d3 2026-07-30 14:34:43,632 - DEBUG - Requesting: http://10.0.0.177:8082/project/43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:34:43,732 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,745 - DEBUG - Requesting: http://10.0.0.177:8082/floating-ip-pool/e64d1466-0290-49e0-a04f-d661a224653d 2026-07-30 14:34:43,753 - DEBUG - Requesting: http://10.0.0.177:8082/domains 2026-07-30 14:34:43,765 - DEBUG - Requesting: http://10.0.0.177:8082/domain/dbe26e84-f679-42d9-bbe3-c4786d1d55d3 2026-07-30 14:34:43,775 - DEBUG - Requesting: http://10.0.0.177:8082/project/43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:34:43,826 - DEBUG - Requesting: http://10.0.0.177:8082/virtual-network/71c3e9b2-c9ce-43fb-803e-8cbcf60264ab 2026-07-30 14:34:43,837 - DEBUG - FIP Pool __fip_pool_public__ found in API Server 2026-07-30 14:34:43,837 - INFO - Verification for FIP pool __fip_pool_public__ in API Server passed 2026-07-30 14:34:43,846 - DEBUG - Requesting: http://10.0.0.177:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-07-30 14:34:43,853 - DEBUG - Control-node Ifmap-view has FIP pool __fip_pool_public__ information 2026-07-30 14:34:43,853 - INFO - Verification for FIP pool __fip_pool_public__ in Control-node passed 2026-07-30 14:34:43,857 - DEBUG - Ingress ctest-nginx-ingress-25422509 not present 2026-07-30 14:34:43,858 - INFO - Creating Ingress ctest-nginx-ingress-25422509 2026-07-30 14:34:43,869 - INFO - Created Ingress ctest-nginx-ingress-25422509 2026-07-30 14:34:43,874 - DEBUG - Cluster IP not yet seen for Ingress ctest-nginx-ingress-25422509 2026-07-30 14:34:46,889 - DEBUG - For Ingress ctest-nginx-ingress-25422509, Cluster IP: 10.47.255.249, External IPs ['92.243.175.124'] 2026-07-30 14:34:46,890 - INFO - Verifications in k8s passed for Ingress ctest-nginx-ingress-25422509 2026-07-30 14:34:46,890 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_LoadbalancerDatabaseList?x=bbac6cc7-1447-488f-9408-2ac538e9074e 2026-07-30 14:34:46,916 - INFO - Ingress ctest-nginx-ingress-25422509 with uuid bbac6cc7-1447-488f-9408-2ac538e9074e found in kube manager 2026-07-30 14:34:46,916 - INFO - Ingress ctest-nginx-ingress-25422509 verification passed 2026-07-30 14:34:46,921 - DEBUG - Pod ctest-busybox-pod-96965957 not present 2026-07-30 14:34:46,922 - INFO - Creating Pod ctest-busybox-pod-96965957 2026-07-30 14:34:46,957 - DEBUG - Pod : ctest-busybox-pod-96965957 UUID is a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:34:46,972 - DEBUG - Pod ctest-nginx-pod-05652946 not in running state.Currently in Pending 2026-07-30 14:34:51,982 - INFO - Pod ctest-nginx-pod-05652946 is in running state.Got IP 10.47.255.251 2026-07-30 14:34:52,041 - DEBUG - Pod ctest-nginx-pod-05652946 has vmi cb78bf40-8c23-11f1-a149-fa163ed675c0 2026-07-30 14:34:52,041 - INFO - Verified pod ctest-nginx-pod-05652946 in contrail-api 2026-07-30 14:34:52,041 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-07-30 14:34:52,056 - DEBUG - VMI cb78bf40-8c23-11f1-a149-fa163ed675c0 is active in agent 10.0.0.38 2026-07-30 14:34:52,057 - INFO - Verified Pod ctest-nginx-pod-05652946 in agent 10.0.0.38 2026-07-30 14:34:52,057 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_VirtualMachineDatabaseList?x=a5e3979c-8dcc-448a-9897-4e5f6db357d1 2026-07-30 14:34:52,065 - INFO - Pod ctest-nginx-pod-05652946 with uuid a5e3979c-8dcc-448a-9897-4e5f6db357d1 found in kube manager 2026-07-30 14:34:52,065 - INFO - Pod ctest-nginx-pod-05652946 verification passed 2026-07-30 14:34:52,158 - DEBUG - [Pod ctest-nginx-pod-05652946] Cmd: echo ctest-nginx-pod-05652946 > /usr/share/nginx/html/index.html, Output: 2026-07-30 14:34:52,166 - INFO - Pod ctest-nginx-pod-05268493 is in running state.Got IP 10.47.255.250 2026-07-30 14:34:52,227 - DEBUG - Pod ctest-nginx-pod-05268493 has vmi cc07b948-8c23-11f1-a149-fa163ed675c0 2026-07-30 14:34:52,227 - INFO - Verified pod ctest-nginx-pod-05268493 in contrail-api 2026-07-30 14:34:52,227 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-07-30 14:34:52,247 - DEBUG - VMI cc07b948-8c23-11f1-a149-fa163ed675c0 is active in agent 10.0.0.38 2026-07-30 14:34:52,247 - INFO - Verified Pod ctest-nginx-pod-05268493 in agent 10.0.0.38 2026-07-30 14:34:52,247 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_VirtualMachineDatabaseList?x=a9585a12-a206-472e-87b0-cd3998ea9fb5 2026-07-30 14:34:52,260 - INFO - Pod ctest-nginx-pod-05268493 with uuid a9585a12-a206-472e-87b0-cd3998ea9fb5 found in kube manager 2026-07-30 14:34:52,260 - INFO - Pod ctest-nginx-pod-05268493 verification passed 2026-07-30 14:34:52,362 - DEBUG - [Pod ctest-nginx-pod-05268493] Cmd: echo ctest-nginx-pod-05268493 > /usr/share/nginx/html/index.html, Output: 2026-07-30 14:34:52,369 - DEBUG - Pod ctest-busybox-pod-96965957 not in running state.Currently in Pending 2026-07-30 14:34:57,382 - DEBUG - Pod ctest-busybox-pod-96965957 not in running state.Currently in Pending 2026-07-30 14:35:02,392 - INFO - Pod ctest-busybox-pod-96965957 is in running state.Got IP 10.47.255.248 2026-07-30 14:35:02,414 - DEBUG - Pod ctest-busybox-pod-96965957 has vmi d0c8681a-8c23-11f1-a149-fa163ed675c0 2026-07-30 14:35:02,414 - INFO - Verified pod ctest-busybox-pod-96965957 in contrail-api 2026-07-30 14:35:02,414 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-07-30 14:35:02,425 - DEBUG - VMI d0c8681a-8c23-11f1-a149-fa163ed675c0 is active in agent 10.0.0.38 2026-07-30 14:35:02,426 - INFO - Verified Pod ctest-busybox-pod-96965957 in agent 10.0.0.38 2026-07-30 14:35:02,426 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_VirtualMachineDatabaseList?x=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:02,431 - INFO - Pod ctest-busybox-pod-96965957 with uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 found in kube manager 2026-07-30 14:35:02,431 - INFO - Pod ctest-busybox-pod-96965957 verification passed 2026-07-30 14:35:02,496 - DEBUG - [Pod ctest-busybox-pod-96965957] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-05268493 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-07-30 14:35:02,496 - DEBUG - [Pod Pod ctest-busybox-pod-96965957] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-07-30 14:35:02,496 - DEBUG - [Pod Pod ctest-busybox-pod-96965957] Cmd output: ctest-nginx-pod-05268493 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-07-30 14:35:02,608 - DEBUG - [Pod ctest-busybox-pod-96965957] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-05652946 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-07-30 14:35:02,608 - DEBUG - [Pod Pod ctest-busybox-pod-96965957] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-07-30 14:35:02,608 - DEBUG - [Pod Pod ctest-busybox-pod-96965957] Cmd output: ctest-nginx-pod-05652946 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-07-30 14:35:02,609 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-nginx-pod-05652946': 1, 'ctest-nginx-pod-05268493': 1} 2026-07-30 14:35:02,609 - WARNING - Nothing to delete parallely 2026-07-30 14:35:02,609 - INFO - Deleting pod default:ctest-busybox-pod-96965957 2026-07-30 14:35:02,624 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:02,633 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:07,634 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:07,642 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:12,643 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:12,652 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:17,653 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:17,665 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:22,667 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:22,675 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:27,676 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:27,683 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:32,686 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:32,695 - WARNING - Pod uuid a65e614d-7130-4ef2-8571-4cc3a5d780b2 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:37,696 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a65e614d-7130-4ef2-8571-4cc3a5d780b2 2026-07-30 14:35:37,704 - DEBUG - Pod a65e614d-7130-4ef2-8571-4cc3a5d780b2 is not in agent 10.0.0.38 VM list 2026-07-30 14:35:37,704 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:d0c8681a-8c23-11f1-a149-fa163ed675c0 2026-07-30 14:35:37,712 - DEBUG - VMI d0c8681a-8c23-11f1-a149-fa163ed675c0 is removed from agent 10.0.0.38 2026-07-30 14:35:37,712 - INFO - Verified that pod ctest-busybox-pod-96965957 is removed in agent 2026-07-30 14:35:37,712 - INFO - Deleting Ingress : ctest-nginx-ingress-25422509 2026-07-30 14:35:37,721 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_LoadbalancerDatabaseList?x=bbac6cc7-1447-488f-9408-2ac538e9074e 2026-07-30 14:35:37,732 - ERROR - Ingress ctest-nginx-ingress-25422509 with uuid bbac6cc7-1447-488f-9408-2ac538e9074e still found in kube manager 2026-07-30 14:35:38,733 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_LoadbalancerDatabaseList?x=bbac6cc7-1447-488f-9408-2ac538e9074e 2026-07-30 14:35:38,738 - ERROR - Ingress ctest-nginx-ingress-25422509 with uuid bbac6cc7-1447-488f-9408-2ac538e9074e still found in kube manager 2026-07-30 14:35:39,740 - DEBUG - Requesting: http://10.0.0.177:8108/Snh_LoadbalancerDatabaseList?x=bbac6cc7-1447-488f-9408-2ac538e9074e 2026-07-30 14:35:39,744 - INFO - Ingress ctest-nginx-ingress-25422509 with uuid bbac6cc7-1447-488f-9408-2ac538e9074e deleted successfully from kube manager 2026-07-30 14:35:39,744 - INFO - Verifications on Ingress %s cleanup passed 2026-07-30 14:35:39,744 - INFO - Deleting the FIP pool __fip_pool_public__ 2026-07-30 14:35:39,819 - DEBUG - Requesting: http://10.0.0.177:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-07-30 14:35:39,825 - DEBUG - Control-node Ifmap-view does not have FIP pool __fip_pool_public__ information 2026-07-30 14:35:39,825 - INFO - Deleting VN __public__ 2026-07-30 14:35:39,965 - DEBUG - Requesting: http://10.0.0.177:8082/routing-instance/0ba96d7e-2fa2-480c-8cc4-3410275d2a18 2026-07-30 14:35:39,976 - DEBUG - Response Code: 404 2026-07-30 14:35:39,976 - DEBUG - Requesting: http://10.0.0.177:8082/domains 2026-07-30 14:35:39,984 - DEBUG - Requesting: http://10.0.0.177:8082/domain/dbe26e84-f679-42d9-bbe3-c4786d1d55d3 2026-07-30 14:35:39,992 - DEBUG - Requesting: http://10.0.0.177:8082/project/43237206-e1eb-4ffd-a906-340c6ed9da44 2026-07-30 14:35:40,039 - INFO - Validated that VN __public__ is not found in API Server 2026-07-30 14:35:40,039 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:k8s-default:__public__ 2026-07-30 14:35:40,049 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:35:40,060 - DEBUG - VN __public__ is not present in Agent 10.0.0.38 2026-07-30 14:35:40,060 - INFO - Validated that VN __public__ is not in any agent 2026-07-30 14:35:40,060 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:35:40,068 - DEBUG - Requesting: http://10.0.0.177:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-07-30 14:35:40,073 - DEBUG - Requesting: http://10.0.0.177:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-07-30 14:35:40,079 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN __public__ info 2026-07-30 14:35:40,079 - INFO - Deleting pod default:ctest-nginx-pod-05268493 2026-07-30 14:35:40,098 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a9585a12-a206-472e-87b0-cd3998ea9fb5 2026-07-30 14:35:40,107 - WARNING - Pod uuid a9585a12-a206-472e-87b0-cd3998ea9fb5 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:45,109 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a9585a12-a206-472e-87b0-cd3998ea9fb5 2026-07-30 14:35:45,116 - DEBUG - Pod a9585a12-a206-472e-87b0-cd3998ea9fb5 is not in agent 10.0.0.38 VM list 2026-07-30 14:35:45,117 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:cc07b948-8c23-11f1-a149-fa163ed675c0 2026-07-30 14:35:45,123 - DEBUG - VMI cc07b948-8c23-11f1-a149-fa163ed675c0 is removed from agent 10.0.0.38 2026-07-30 14:35:45,123 - INFO - Verified that pod ctest-nginx-pod-05268493 is removed in agent 2026-07-30 14:35:45,123 - INFO - Deleting pod default:ctest-nginx-pod-05652946 2026-07-30 14:35:45,137 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a5e3979c-8dcc-448a-9897-4e5f6db357d1 2026-07-30 14:35:45,147 - WARNING - Pod uuid a5e3979c-8dcc-448a-9897-4e5f6db357d1 is still seen in agent 10.0.0.38 VM list 2026-07-30 14:35:50,148 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VmListReq?uuid=a5e3979c-8dcc-448a-9897-4e5f6db357d1 2026-07-30 14:35:50,155 - DEBUG - Pod a5e3979c-8dcc-448a-9897-4e5f6db357d1 is not in agent 10.0.0.38 VM list 2026-07-30 14:35:50,155 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:cb78bf40-8c23-11f1-a149-fa163ed675c0 2026-07-30 14:35:50,162 - DEBUG - VMI cb78bf40-8c23-11f1-a149-fa163ed675c0 is removed from agent 10.0.0.38 2026-07-30 14:35:50,162 - INFO - Verified that pod ctest-nginx-pod-05652946 is removed in agent 2026-07-30 14:35:50,162 - INFO - Deleting service : ctest-nginx-svc-08200534 2026-07-30 14:35:50,469 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq? 2026-07-30 14:35:50,478 - DEBUG - No XMPP flaps were noticed during the test