2025-12-11 01:47:05,019 - DEBUG - [10.0.0.22]: Running cmd : hostname
2025-12-11 01:47:05,288 - DEBUG - Output : cn-jenkins-deploy-platform-k8s-manifests-278-1
2025-12-11 01:47:05,289 - DEBUG - [10.0.0.22]: Running cmd : hostname -f
2025-12-11 01:47:05,407 - DEBUG - Output : cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local
2025-12-11 01:47:05,408 - DEBUG - [10.0.0.22]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}'
2025-12-11 01:47:05,594 - DEBUG - Output : NAMES
contrail_test_5sS8Cnvj1
k8s_autoscaler_dns-autoscaler-595558c478-xq7gq_kube-system_e519f99f-b8c8-4e25-9a4d-88016d9c6fee_0
k8s_coredns_coredns-77f7cc69db-q4t5h_kube-system_71282a5e-850e-46cf-8f46-1d3895c2b086_0
k8s_opensdn-controller-config-nodemgr_opensdn-controller-config-25hcb_kube-system_aba02c6e-1a32-476e-a3c8-407f4e962255_0
k8s_opensdn-controller-config-svcmonitor_opensdn-controller-config-25hcb_kube-system_aba02c6e-1a32-476e-a3c8-407f4e962255_0
k8s_opensdn-controller-config-schema_opensdn-controller-config-25hcb_kube-system_aba02c6e-1a32-476e-a3c8-407f4e962255_0
k8s_opensdn-controller-nodemgr_opensdn-controller-control-9l4g4_kube-system_b860c42e-5b0a-46bf-9f44-dac6d72381d5_0
k8s_opensdn-controller-control-named_opensdn-controller-control-9l4g4_kube-system_b860c42e-5b0a-46bf-9f44-dac6d72381d5_0
k8s_opensdn-analytics-query-engine_opensdn-analyticsdb-rvmkn_kube-system_400ed156-2bb5-4218-a8af-70f7776c31a2_0
k8s_opensdn-analytics-nodemgr_opensdn-analytics-lp4xg_kube-system_efe606c4-2b62-4bef-bfd0-e5a3c0daa4f1_0
k8s_opensdn-analytics-collector_opensdn-analytics-lp4xg_kube-system_efe606c4-2b62-4bef-bfd0-e5a3c0daa4f1_0
k8s_opensdn-controller-config-devicemgr_opensdn-controller-config-25hcb_kube-system_aba02c6e-1a32-476e-a3c8-407f4e962255_0
k8s_opensdn-analytics-snmp-nodemgr_opensdn-analytics-snmp-tnvkp_kube-system_ec13ed70-2791-43b4-8331-13b6f4697ff0_0
k8s_opensdn-analytics-snmp-topology_opensdn-analytics-snmp-tnvkp_kube-system_ec13ed70-2791-43b4-8331-13b6f4697ff0_0
k8s_opensdn-controller-webui-web_opensdn-controller-webui-kdxtk_kube-system_33d257b4-a1d7-4569-9d96-338a942c577b_0
k8s_opensdn-controller-control-dns_opensdn-controller-control-9l4g4_kube-system_b860c42e-5b0a-46bf-9f44-dac6d72381d5_0
k8s_opensdn-analyticsdb_opensdn-analyticsdb-rvmkn_kube-system_400ed156-2bb5-4218-a8af-70f7776c31a2_0
k8s_opensdn-analytics-alarm-nodemgr_opensdn-analytics-alarm-7s8t9_kube-system_b436b8fe-95d7-4a11-900d-6cea04ea3d11_0
k8s_opensdn-analytics-alarm-gen_opensdn-analytics-alarm-7s8t9_kube-system_b436b8fe-95d7-4a11-900d-6cea04ea3d11_0
k8s_opensdn-config-database-nodemgr_opensdn-configdb-jzfns_kube-system_61568a79-aa97-407d-80df-74a7a6ed7c5d_0
k8s_opensdn-analytics-api_opensdn-analytics-lp4xg_kube-system_efe606c4-2b62-4bef-bfd0-e5a3c0daa4f1_0
k8s_opensdn-controller-config-api_opensdn-controller-config-25hcb_kube-system_aba02c6e-1a32-476e-a3c8-407f4e962255_0
k8s_opensdn-analytics-snmp-collector_opensdn-analytics-snmp-tnvkp_kube-system_ec13ed70-2791-43b4-8331-13b6f4697ff0_0
k8s_opensdn-controller-webui-job_opensdn-controller-webui-kdxtk_kube-system_33d257b4-a1d7-4569-9d96-338a942c577b_0
k8s_opensdn-controller-control_opensdn-controller-control-9l4g4_kube-system_b860c42e-5b0a-46bf-9f44-dac6d72381d5_0
k8s_opensdn-analyticsdb-nodemgr_opensdn-analyticsdb-rvmkn_kube-system_400ed156-2bb5-4218-a8af-70f7776c31a2_0
k8s_kafka_opensdn-analytics-alarm-7s8t9_kube-system_b436b8fe-95d7-4a11-900d-6cea04ea3d11_0
k8s_redis_redis-cj9gs_kube-system_81c0828f-c1ac-4f35-ac1b-c2ef144de564_0
k8s_opensdn-agent-nodemgr_opensdn-agent-flm82_kube-system_816a9bdb-539b-45e6-8133-193900fa5fac_0
k8s_rabbitmq_rabbitmq-vkrjg_kube-system_02e85652-04f6-453c-8f3e-55246974c429_0
k8s_opensdn-configdb_opensdn-configdb-jzfns_kube-system_61568a79-aa97-407d-80df-74a7a6ed7c5d_0
k8s_config-zookeeper_config-zookeeper-rl485_kube-system_23690894-2ab8-479d-92a7-1a5cbca79e0f_0
k8s_opensdn-vrouter-agent_opensdn-agent-flm82_kube-system_816a9bdb-539b-45e6-8133-193900fa5fac_0
k8s_opensdn-kube-manager_opensdn-kube-manager-6krlv_kube-system_9e3000bc-d0b3-4986-9bff-3f5adcd3df4b_0
k8s_kube-proxy_kube-proxy-gbrnc_kube-system_51924719-9377-4a30-9e27-2300ab342f45_0
k8s_kube-scheduler_kube-scheduler-cn-jenkins-deploy-platform-k8s-manifests-278-1_kube-system_a7b0742af8779b349f62adf60b4d5f52_1
k8s_kube-controller-manager_kube-controller-manager-cn-jenkins-deploy-platform-k8s-manifests-278-1_kube-system_2e25149b887c2b4171d9f967b68690b3_1
k8s_kube-apiserver_kube-apiserver-cn-jenkins-deploy-platform-k8s-manifests-278-1_kube-system_05c3d80e3ddbb013dd2c54c6c64fb9ce_0
2025-12-11 01:47:05,594 - DEBUG - [10.0.0.22]: Running cmd : ip -4 -o addr show | awk '{print $4}'
2025-12-11 01:47:05,714 - DEBUG - Output : 127.0.0.1/8
172.17.0.1/16
10.96.0.1/32
10.96.0.3/32
10.0.0.22/24
2025-12-11 01:47:05,714 - DEBUG - [10.0.0.22]: Running cmd : getent hosts 10.0.0.22 | head -n 1 | awk '{print $2}'
2025-12-11 01:47:05,849 - DEBUG - Output : cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local
2025-12-11 01:47:05,849 - DEBUG - [10.0.0.22]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}'
2025-12-11 01:47:05,959 - DEBUG - Output : 10.0.0.22/24
2025-12-11 01:47:06,315 - INFO - Using existing project ['default-domain', 'k8s-default'](ffb7b5fe-a452-4bc4-b32c-948532be51b8)
2025-12-11 01:47:06,440 - INFO - Using existing project ['default-domain', 'k8s-default'](ffb7b5fe-a452-4bc4-b32c-948532be51b8)
2025-12-11 01:47:06,561 - INFO - ================================================================================
2025-12-11 01:47:06,561 - INFO - STARTING TEST : test_ingress_ip_assignment
2025-12-11 01:47:06,561 - INFO - TEST DESCRIPTION :
Verify that Ingress gets a CLuster IP which is reachable to Pods in same
namespace. Also verify that a Floating IP is assigned to the Ingress
from the Public FIP poo.
Steps:
1. Create a service with 2 pods running nginx
2. Create an ingress out of this service
3. From another Pod do a wget on the ingress Cluster ip
Validate that Ingress get a IP from Public FIP pool which might/might not be accessible.
Validate that service and its loadbalancing work
2025-12-11 01:47:06,833 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq?
2025-12-11 01:47:06,839 - DEBUG - Nothing to compare xmpp stats {'10.0.0.22': {'10.0.0.22': '1'}} with
2025-12-11 01:47:06,839 - INFO - Initial checks done. Running the testcase now
2025-12-11 01:47:06,839 - INFO -
2025-12-11 01:47:06,843 - INFO - Namespace default already exists
2025-12-11 01:47:06,844 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it k8s_opensdn-kube-manager_opensdn-kube-manager-6krlv_kube-system_9e3000bc-d0b3-4986-9bff-3f5adcd3df4b_0 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf'
2025-12-11 01:47:07,018 - DEBUG - Output : cluster_project={}
2025-12-11 01:47:07,090 - INFO - Project default-domain:k8s-default for Namespace default is seen in contrail-api
2025-12-11 01:47:07,090 - DEBUG - Project uuid in Contrail is ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:47:07,091 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it k8s_opensdn-kube-manager_opensdn-kube-manager-6krlv_kube-system_9e3000bc-d0b3-4986-9bff-3f5adcd3df4b_0 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN'
2025-12-11 01:47:07,237 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3
2025-12-11 01:47:07,238 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NamespaceDatabaseList?namespace_uuid=ba16933c-21b5-4904-869f-7e80779f8c23
2025-12-11 01:47:07,241 - INFO - Namespace default with uuid ba16933c-21b5-4904-869f-7e80779f8c23 found in kube manager
2025-12-11 01:47:07,241 - INFO - Namespace default verification passed
2025-12-11 01:47:07,245 - DEBUG - Service ctest-nginx-svc-36727347 not present
2025-12-11 01:47:07,245 - INFO - Creating service ctest-nginx-svc-36727347
2025-12-11 01:47:07,258 - INFO - Created Service ctest-nginx-svc-36727347
2025-12-11 01:47:07,262 - DEBUG - Pod ctest-nginx-pod-17213836 not present
2025-12-11 01:47:07,263 - INFO - Creating Pod ctest-nginx-pod-17213836
2025-12-11 01:47:07,291 - DEBUG - Pod : ctest-nginx-pod-17213836 UUID is 4826c398-554f-4fb1-9b3a-376a5d1fe7d4
2025-12-11 01:47:07,294 - DEBUG - Pod ctest-nginx-pod-83233696 not present
2025-12-11 01:47:07,294 - INFO - Creating Pod ctest-nginx-pod-83233696
2025-12-11 01:47:07,318 - DEBUG - Pod : ctest-nginx-pod-83233696 UUID is bfb8dd62-7a9f-49d6-8fec-51fc3fa01836
2025-12-11 01:47:07,575 - INFO - Created VN __public__, UUID :c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:07,821 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2025-12-11 01:47:07,827 - DEBUG - Requesting: http://10.0.0.22:8082/domain/3e6c031c-0471-491b-9a50-8521c8f5d3f3
2025-12-11 01:47:07,845 - DEBUG - Requesting: http://10.0.0.22:8082/project/ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:47:07,893 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:07,919 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:07,928 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:07,934 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:07,949 - DEBUG - Route Targets: []
2025-12-11 01:47:07,949 - DEBUG - RT names not yet present for VN __public__
2025-12-11 01:47:12,950 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2025-12-11 01:47:12,955 - DEBUG - Requesting: http://10.0.0.22:8082/domain/3e6c031c-0471-491b-9a50-8521c8f5d3f3
2025-12-11 01:47:12,960 - DEBUG - Requesting: http://10.0.0.22:8082/project/ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:47:12,997 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,005 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,017 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:13,049 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:13,055 - DEBUG - Requesting: http://10.0.0.22:8082/route-target/60469eb6-b87d-4c29-8ed8-24eb4005afed
2025-12-11 01:47:13,061 - DEBUG - Route Targets: ['target:64512:8000006']
2025-12-11 01:47:13,061 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,069 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:13,082 - INFO - Verified VN network id 11 for VN c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,082 - INFO - Verifications in API Server for VN __public__ passed
2025-12-11 01:47:13,082 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,090 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:13,095 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:47:13,100 - DEBUG - Requesting: http://10.0.0.22:8082/route-target/60469eb6-b87d-4c29-8ed8-24eb4005afed
2025-12-11 01:47:13,106 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__
2025-12-11 01:47:13,114 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:k8s-default:__public__', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'cloud-admin', 'owner-access': '7', 'group': 'cloud-admin-group', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14248433164851954455', 'uuid-lslong': '11268112846809744158'}, 'enable': 'true', 'created': '2025-12-11T01:47:07', 'last-modified': '2025-12-11T01:47:07', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.407643'}
2025-12-11 01:47:13,114 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:47:13,122 - DEBUG - Route Targets: ['target:64512:8000006']
2025-12-11 01:47:13,122 - INFO - On all control nodes, Config, RI and RT verification for VN __public__ passed
2025-12-11 01:47:13,123 - DEBUG - ====Verifying policy data for __public__ in API_Server ======
2025-12-11 01:47:13,123 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2025-12-11 01:47:13,130 - DEBUG - Requesting: http://10.0.0.22:8082/domain/3e6c031c-0471-491b-9a50-8521c8f5d3f3
2025-12-11 01:47:13,136 - DEBUG - Requesting: http://10.0.0.22:8082/project/ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:47:13,164 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,172 - DEBUG - =>VN __public__ has no policy to be verified
2025-12-11 01:47:13,172 - DEBUG - Verifying the vn in opserver
2025-12-11 01:47:13,172 - DEBUG - Verifying the default-domain:k8s-default:__public__ virtual network link through opserver 10.0.0.22
2025-12-11 01:47:13,172 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks
2025-12-11 01:47:13,178 - DEBUG - vn link and name as {'name': 'default-domain:k8s-default:__public__', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:k8s-default:__public__?flat'}
2025-12-11 01:47:13,178 - INFO - Validated that VN default-domain:k8s-default:__public__ is found in opserver
2025-12-11 01:47:13,178 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:47:13,185 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:47:13,190 - DEBUG - Do not have enough data to verify VN in agent
2025-12-11 01:47:13,190 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:47:13,195 - DEBUG - VRF ids for VN __public__: {}
2025-12-11 01:47:13,359 - INFO - Creating Floating IP pool __fip_pool_public__ in API Server
2025-12-11 01:47:13,568 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2025-12-11 01:47:13,574 - DEBUG - Requesting: http://10.0.0.22:8082/domain/3e6c031c-0471-491b-9a50-8521c8f5d3f3
2025-12-11 01:47:13,580 - DEBUG - Requesting: http://10.0.0.22:8082/project/ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:47:13,612 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,632 - DEBUG - Requesting: http://10.0.0.22:8082/floating-ip-pool/748606ba-8cb7-4270-9295-54749354c406
2025-12-11 01:47:13,640 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2025-12-11 01:47:13,657 - DEBUG - Requesting: http://10.0.0.22:8082/domain/3e6c031c-0471-491b-9a50-8521c8f5d3f3
2025-12-11 01:47:13,682 - DEBUG - Requesting: http://10.0.0.22:8082/project/ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:47:13,760 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/c5bc9a78-d4e5-4317-9c60-60eee31c431e
2025-12-11 01:47:13,778 - DEBUG - FIP Pool __fip_pool_public__ found in API Server
2025-12-11 01:47:13,778 - INFO - Verification for FIP pool __fip_pool_public__ in API Server passed
2025-12-11 01:47:13,789 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__
2025-12-11 01:47:13,798 - DEBUG - Control-node Ifmap-view has FIP pool __fip_pool_public__ information
2025-12-11 01:47:13,798 - INFO - Verification for FIP pool __fip_pool_public__ in Control-node passed
2025-12-11 01:47:13,801 - DEBUG - Ingress ctest-nginx-ingress-19208256 not present
2025-12-11 01:47:13,802 - INFO - Creating Ingress ctest-nginx-ingress-19208256
2025-12-11 01:47:13,811 - INFO - Created Ingress ctest-nginx-ingress-19208256
2025-12-11 01:47:13,824 - DEBUG - Cluster IP not yet seen for Ingress ctest-nginx-ingress-19208256
2025-12-11 01:47:16,830 - DEBUG - For Ingress ctest-nginx-ingress-19208256, Cluster IP: 10.47.255.248, External IPs ['16.82.222.60']
2025-12-11 01:47:16,830 - INFO - Verifications in k8s passed for Ingress ctest-nginx-ingress-19208256
2025-12-11 01:47:16,830 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=9e5dae6e-d52b-4db6-bae7-dc944c7f728d
2025-12-11 01:47:16,837 - INFO - Ingress ctest-nginx-ingress-19208256 with uuid 9e5dae6e-d52b-4db6-bae7-dc944c7f728d found in kube manager
2025-12-11 01:47:16,837 - INFO - Ingress ctest-nginx-ingress-19208256 verification passed
2025-12-11 01:47:16,840 - DEBUG - Pod ctest-busybox-pod-19799159 not present
2025-12-11 01:47:16,840 - INFO - Creating Pod ctest-busybox-pod-19799159
2025-12-11 01:47:16,873 - DEBUG - Pod : ctest-busybox-pod-19799159 UUID is 57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:16,882 - DEBUG - Pod ctest-nginx-pod-17213836 not in running state.Currently in Pending
2025-12-11 01:47:21,889 - DEBUG - Pod ctest-nginx-pod-17213836 not in running state.Currently in Pending
2025-12-11 01:47:26,899 - INFO - Pod ctest-nginx-pod-17213836 is in running state.Got IP 10.47.255.250
2025-12-11 01:47:26,962 - DEBUG - Pod ctest-nginx-pod-17213836 has vmi 4e9e5a12-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:47:26,962 - INFO - Verified pod ctest-nginx-pod-17213836 in contrail-api
2025-12-11 01:47:26,962 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:47:26,972 - DEBUG - VMI 4e9e5a12-d633-11f0-afdd-0242e7b680b1 is active in agent 10.0.0.22
2025-12-11 01:47:26,972 - INFO - Verified Pod ctest-nginx-pod-17213836 in agent 10.0.0.22
2025-12-11 01:47:26,972 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=4826c398-554f-4fb1-9b3a-376a5d1fe7d4
2025-12-11 01:47:26,976 - INFO - Pod ctest-nginx-pod-17213836 with uuid 4826c398-554f-4fb1-9b3a-376a5d1fe7d4 found in kube manager
2025-12-11 01:47:26,976 - INFO - Pod ctest-nginx-pod-17213836 verification passed
2025-12-11 01:47:27,072 - DEBUG - [Pod ctest-nginx-pod-17213836] Cmd: echo ctest-nginx-pod-17213836 > /usr/share/nginx/html/index.html, Output:
2025-12-11 01:47:27,079 - INFO - Pod ctest-nginx-pod-83233696 is in running state.Got IP 10.47.255.249
2025-12-11 01:47:27,130 - DEBUG - Pod ctest-nginx-pod-83233696 has vmi 4f4bad02-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:47:27,130 - INFO - Verified pod ctest-nginx-pod-83233696 in contrail-api
2025-12-11 01:47:27,130 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:47:27,141 - DEBUG - VMI 4f4bad02-d633-11f0-afdd-0242e7b680b1 is active in agent 10.0.0.22
2025-12-11 01:47:27,141 - INFO - Verified Pod ctest-nginx-pod-83233696 in agent 10.0.0.22
2025-12-11 01:47:27,141 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=bfb8dd62-7a9f-49d6-8fec-51fc3fa01836
2025-12-11 01:47:27,144 - INFO - Pod ctest-nginx-pod-83233696 with uuid bfb8dd62-7a9f-49d6-8fec-51fc3fa01836 found in kube manager
2025-12-11 01:47:27,144 - INFO - Pod ctest-nginx-pod-83233696 verification passed
2025-12-11 01:47:27,217 - DEBUG - [Pod ctest-nginx-pod-83233696] Cmd: echo ctest-nginx-pod-83233696 > /usr/share/nginx/html/index.html, Output:
2025-12-11 01:47:27,224 - INFO - Pod ctest-busybox-pod-19799159 is in running state.Got IP 10.47.255.247
2025-12-11 01:47:27,275 - DEBUG - Pod ctest-busybox-pod-19799159 has vmi 542d4a7e-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:47:27,275 - INFO - Verified pod ctest-busybox-pod-19799159 in contrail-api
2025-12-11 01:47:27,275 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:47:27,286 - DEBUG - VMI 542d4a7e-d633-11f0-afdd-0242e7b680b1 is active in agent 10.0.0.22
2025-12-11 01:47:27,286 - INFO - Verified Pod ctest-busybox-pod-19799159 in agent 10.0.0.22
2025-12-11 01:47:27,286 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:27,291 - INFO - Pod ctest-busybox-pod-19799159 with uuid 57a9ea95-e36a-431d-a494-84896cc402b0 found in kube manager
2025-12-11 01:47:27,291 - INFO - Pod ctest-busybox-pod-19799159 verification passed
2025-12-11 01:47:27,368 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,368 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,368 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,447 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,447 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,447 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,545 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
ctest-nginx-pod-17213836
2025-12-11 01:47:27,545 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,545 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
ctest-nginx-pod-17213836
2025-12-11 01:47:27,629 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,629 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,629 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,700 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,700 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,700 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,773 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
ctest-nginx-pod-17213836
2025-12-11 01:47:27,773 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,773 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
ctest-nginx-pod-17213836
2025-12-11 01:47:27,856 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
ctest-nginx-pod-17213836
2025-12-11 01:47:27,856 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,857 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
ctest-nginx-pod-17213836
2025-12-11 01:47:27,924 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,924 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,924 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,988 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:27,989 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:27,989 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:28,051 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:28,051 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:28,051 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-17213836
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:28,051 - WARNING - No http hit seen for one or more pods.Pls check. Hits: {'ctest-nginx-pod-17213836': 10, 'ctest-nginx-pod-83233696': 0}
2025-12-11 01:47:30,126 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-83233696
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:30,126 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:30,126 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
writing to stdout
ctest-nginx-pod-83233696
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:30,204 - DEBUG - [Pod ctest-busybox-pod-19799159] Cmd: wget http://10.47.255.248:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:30,205 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd wget http://10.47.255.248:80/ -O - -T 5 -t 1 passed
2025-12-11 01:47:30,205 - DEBUG - [Pod Pod ctest-busybox-pod-19799159] Cmd output: Connecting to 10.47.255.248:80 (10.47.255.248:80)
ctest-nginx-pod-17213836
writing to stdout
- 100% |********************************| 25 0:00:00 ETA
written to stdout
2025-12-11 01:47:30,205 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-nginx-pod-17213836': 1, 'ctest-nginx-pod-83233696': 1}
2025-12-11 01:47:30,205 - WARNING - Nothing to delete parallely
2025-12-11 01:47:30,205 - INFO - Deleting pod default:ctest-busybox-pod-19799159
2025-12-11 01:47:30,219 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:30,223 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:47:35,224 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:35,230 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:47:40,231 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:40,238 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:47:45,238 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:45,245 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:47:50,246 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:50,252 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:47:55,253 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:47:55,259 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:48:00,260 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:48:00,264 - WARNING - Pod uuid 57a9ea95-e36a-431d-a494-84896cc402b0 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:48:05,265 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=57a9ea95-e36a-431d-a494-84896cc402b0
2025-12-11 01:48:05,270 - DEBUG - Pod 57a9ea95-e36a-431d-a494-84896cc402b0 is not in agent 10.0.0.22 VM list
2025-12-11 01:48:05,270 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:542d4a7e-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:48:05,275 - DEBUG - VMI 542d4a7e-d633-11f0-afdd-0242e7b680b1 is removed from agent 10.0.0.22
2025-12-11 01:48:05,276 - INFO - Verified that pod ctest-busybox-pod-19799159 is removed in agent
2025-12-11 01:48:05,276 - INFO - Deleting Ingress : ctest-nginx-ingress-19208256
2025-12-11 01:48:05,284 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=9e5dae6e-d52b-4db6-bae7-dc944c7f728d
2025-12-11 01:48:05,296 - ERROR - Ingress ctest-nginx-ingress-19208256 with uuid 9e5dae6e-d52b-4db6-bae7-dc944c7f728d still found in kube manager
2025-12-11 01:48:06,297 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=9e5dae6e-d52b-4db6-bae7-dc944c7f728d
2025-12-11 01:48:06,305 - ERROR - Ingress ctest-nginx-ingress-19208256 with uuid 9e5dae6e-d52b-4db6-bae7-dc944c7f728d still found in kube manager
2025-12-11 01:48:07,306 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=9e5dae6e-d52b-4db6-bae7-dc944c7f728d
2025-12-11 01:48:07,309 - INFO - Ingress ctest-nginx-ingress-19208256 with uuid 9e5dae6e-d52b-4db6-bae7-dc944c7f728d deleted successfully from kube manager
2025-12-11 01:48:07,309 - INFO - Verifications on Ingress %s cleanup passed
2025-12-11 01:48:07,309 - INFO - Deleting the FIP pool __fip_pool_public__
2025-12-11 01:48:07,412 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__
2025-12-11 01:48:07,417 - DEBUG - Control-node Ifmap-view does not have FIP pool __fip_pool_public__ information
2025-12-11 01:48:07,417 - INFO - Deleting VN __public__
2025-12-11 01:48:07,506 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/cd0a1997-f7ca-40ab-bd59-5ec72a898935
2025-12-11 01:48:07,510 - DEBUG - Response Code: 404
2025-12-11 01:48:07,511 - DEBUG - Requesting: http://10.0.0.22:8082/domains
2025-12-11 01:48:07,515 - DEBUG - Requesting: http://10.0.0.22:8082/domain/3e6c031c-0471-491b-9a50-8521c8f5d3f3
2025-12-11 01:48:07,524 - DEBUG - Requesting: http://10.0.0.22:8082/project/ffb7b5fe-a452-4bc4-b32c-948532be51b8
2025-12-11 01:48:07,584 - INFO - Validated that VN __public__ is not found in API Server
2025-12-11 01:48:07,584 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VnListReq?name=default-domain:k8s-default:__public__
2025-12-11 01:48:07,590 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:48:07,597 - DEBUG - VN __public__ is not present in Agent 10.0.0.22
2025-12-11 01:48:07,597 - INFO - Validated that VN __public__ is not in any agent
2025-12-11 01:48:07,597 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:48:07,602 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__
2025-12-11 01:48:07,611 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__
2025-12-11 01:48:07,617 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN __public__ info
2025-12-11 01:48:07,618 - INFO - Deleting pod default:ctest-nginx-pod-83233696
2025-12-11 01:48:07,628 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=bfb8dd62-7a9f-49d6-8fec-51fc3fa01836
2025-12-11 01:48:07,636 - WARNING - Pod uuid bfb8dd62-7a9f-49d6-8fec-51fc3fa01836 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:48:12,637 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=bfb8dd62-7a9f-49d6-8fec-51fc3fa01836
2025-12-11 01:48:12,643 - DEBUG - Pod bfb8dd62-7a9f-49d6-8fec-51fc3fa01836 is not in agent 10.0.0.22 VM list
2025-12-11 01:48:12,643 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4f4bad02-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:48:12,649 - DEBUG - VMI 4f4bad02-d633-11f0-afdd-0242e7b680b1 is removed from agent 10.0.0.22
2025-12-11 01:48:12,649 - INFO - Verified that pod ctest-nginx-pod-83233696 is removed in agent
2025-12-11 01:48:12,651 - INFO - Deleting pod default:ctest-nginx-pod-17213836
2025-12-11 01:48:12,679 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=4826c398-554f-4fb1-9b3a-376a5d1fe7d4
2025-12-11 01:48:12,688 - WARNING - Pod uuid 4826c398-554f-4fb1-9b3a-376a5d1fe7d4 is still seen in agent 10.0.0.22 VM list
2025-12-11 01:48:17,689 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=4826c398-554f-4fb1-9b3a-376a5d1fe7d4
2025-12-11 01:48:17,695 - DEBUG - Pod 4826c398-554f-4fb1-9b3a-376a5d1fe7d4 is not in agent 10.0.0.22 VM list
2025-12-11 01:48:17,695 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4e9e5a12-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:48:17,701 - DEBUG - VMI 4e9e5a12-d633-11f0-afdd-0242e7b680b1 is removed from agent 10.0.0.22
2025-12-11 01:48:17,701 - INFO - Verified that pod ctest-nginx-pod-17213836 is removed in agent
2025-12-11 01:48:17,701 - INFO - Deleting service : ctest-nginx-svc-36727347
2025-12-11 01:48:18,241 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq?
2025-12-11 01:48:18,246 - DEBUG - No XMPP flaps were noticed during the test
2025-12-11 01:48:18,247 - INFO - END TEST : test_ingress_ip_assignment : PASSED[0:01:12]
2025-12-11 01:48:18,247 - INFO - --------------------------------------------------------------------------------
2025-12-11 01:48:24,605 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it k8s_opensdn-kube-manager_opensdn-kube-manager-6krlv_kube-system_9e3000bc-d0b3-4986-9bff-3f5adcd3df4b_0 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf'
2025-12-11 01:48:24,768 - DEBUG - Output : cluster_project={}
2025-12-11 01:48:24,847 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it k8s_opensdn-kube-manager_opensdn-kube-manager-6krlv_kube-system_9e3000bc-d0b3-4986-9bff-3f5adcd3df4b_0 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN'
2025-12-11 01:48:24,995 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3
2025-12-11 01:48:24,999 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it k8s_opensdn-kube-manager_opensdn-kube-manager-6krlv_kube-system_9e3000bc-d0b3-4986-9bff-3f5adcd3df4b_0 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf'
2025-12-11 01:48:25,178 - DEBUG - Output : cluster_project={}
2025-12-11 01:48:35,383 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:40,470 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:40,586 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:45,666 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:45,785 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:50,885 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:51,020 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:51,143 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,
2025-12-11 01:48:51,462 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq?
2025-12-11 01:48:54,493 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_ApplicationPolicySetReq?
2025-12-11 01:48:54,499 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AclReq?
2025-12-11 01:48:54,524 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AclReq?
2025-12-11 01:48:56,547 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AclReq?
2025-12-11 01:48:58,565 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AclReq?
2025-12-11 01:52:00,638 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_AgentXmppConnectionStatusReq?
2025-12-11 01:52:00,723 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=ae7508b3-9d48-4766-86e3-dcaa2149f4bc
2025-12-11 01:52:00,737 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=86176497-d361-4f1a-8048-bcef2f0dcb08
2025-12-11 01:52:00,748 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e2960d1d-96b3-43a0-ad7e-6a7517e67d99
2025-12-11 01:52:00,757 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:00,763 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:00,768 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=2ff5d5a7-2c79-4fef-8b41-5d83eaa315b4
2025-12-11 01:52:00,772 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:00,779 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:05,806 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:05,807 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:05,808 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:05,809 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=2ff5d5a7-2c79-4fef-8b41-5d83eaa315b4
2025-12-11 01:52:05,810 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:05,812 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=ae7508b3-9d48-4766-86e3-dcaa2149f4bc
2025-12-11 01:52:05,812 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=86176497-d361-4f1a-8048-bcef2f0dcb08
2025-12-11 01:52:05,813 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e2960d1d-96b3-43a0-ad7e-6a7517e67d99
2025-12-11 01:52:05,826 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7d9840bc-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:05,831 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7e86f482-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:05,832 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7f3edac0-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:05,833 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:7ce2f342-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:10,829 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:10,831 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:10,835 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:10,836 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:15,844 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:15,846 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:15,847 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:15,848 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:20,859 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:20,861 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:20,862 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:20,863 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:25,866 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:25,875 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:25,876 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:25,877 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:30,872 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:30,886 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:30,888 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:30,890 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:35,878 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=64a9cae9-f0fe-466d-b7ac-dc8105b24df5
2025-12-11 01:52:35,883 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:81b570b6-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:35,899 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=e7e64dca-2f3b-4b6d-a1c0-b3cd972891b1
2025-12-11 01:52:35,900 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=c5ea9ef8-a693-4363-830e-d83aef6c30a1
2025-12-11 01:52:35,901 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_VmListReq?uuid=44001480-7d0c-4090-9ca3-da2338c931ad
2025-12-11 01:52:35,911 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:802ff8c4-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:35,913 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:80f3efd6-d633-11f0-afdd-0242e7b680b1
2025-12-11 01:52:35,915 - DEBUG - Requesting: http://10.0.0.22:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:82a415c2-d633-11f0-afdd-0242e7b680b1