2025-11-10 18:35:17,057 - INFO - Using existing project ['default-domain', 'k8s-default'](0c58d16f-d939-4c5d-b747-9e8edf66b6be) 2025-11-10 18:35:17,077 - DEBUG - Namespace ctest-new-default-69240602 not present 2025-11-10 18:35:17,089 - INFO - Created namespace ctest-new-default-69240602 2025-11-10 18:35:20,098 - DEBUG - Namespace ctest-non-default-59661362 not present 2025-11-10 18:35:20,110 - INFO - Created namespace ctest-non-default-59661362 2025-11-10 18:35:23,140 - DEBUG - Pod ctest-pod-22080227 not present 2025-11-10 18:35:23,141 - INFO - Creating Pod ctest-pod-22080227 2025-11-10 18:35:23,239 - DEBUG - Pod : ctest-pod-22080227 UUID is 4daac3bf-f63d-4cef-bc21-358e47551a58 2025-11-10 18:35:23,252 - DEBUG - Pod ctest-pod-30809959 not present 2025-11-10 18:35:23,253 - INFO - Creating Pod ctest-pod-30809959 2025-11-10 18:35:23,361 - DEBUG - Pod : ctest-pod-30809959 UUID is 42be9da6-2e23-4c07-b722-c28c5824c31e 2025-11-10 18:35:23,406 - DEBUG - Pod ctest-pod-71702726 not present 2025-11-10 18:35:23,407 - INFO - Creating Pod ctest-pod-71702726 2025-11-10 18:35:23,523 - DEBUG - Pod : ctest-pod-71702726 UUID is 37ac037f-22dc-48ad-9872-55ff44d6982e 2025-11-10 18:35:23,535 - DEBUG - Pod ctest-pod-16506760 not present 2025-11-10 18:35:23,536 - INFO - Creating Pod ctest-pod-16506760 2025-11-10 18:35:23,572 - DEBUG - Pod : ctest-pod-16506760 UUID is 7f6add15-b79f-4d7f-89b4-38cccb627aca 2025-11-10 18:35:23,579 - DEBUG - Pod ctest-pod-55044620 not present 2025-11-10 18:35:23,580 - INFO - Creating Pod ctest-pod-55044620 2025-11-10 18:35:23,612 - DEBUG - Pod : ctest-pod-55044620 UUID is 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a 2025-11-10 18:35:23,614 - DEBUG - Pod ctest-pod-50111810 not present 2025-11-10 18:35:23,615 - INFO - Creating Pod ctest-pod-50111810 2025-11-10 18:35:23,655 - DEBUG - Pod : ctest-pod-50111810 UUID is bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 2025-11-10 18:35:23,658 - DEBUG - Pod ctest-pod-27738480 not present 2025-11-10 18:35:23,659 - INFO - Creating Pod ctest-pod-27738480 2025-11-10 18:35:23,696 - DEBUG - Pod : ctest-pod-27738480 UUID is 0d84ce4a-7d70-4441-af0e-0681403d41cb 2025-11-10 18:35:23,699 - DEBUG - Pod ctest-pod-68851038 not present 2025-11-10 18:35:23,700 - INFO - Creating Pod ctest-pod-68851038 2025-11-10 18:35:23,753 - DEBUG - Pod : ctest-pod-68851038 UUID is 8d75cdcd-fc54-4344-88fb-5f238d79a46e 2025-11-10 18:35:23,950 - INFO - Project default-domain:k8s-ctest-new-default-69240602 for Namespace ctest-new-default-69240602 is seen in contrail-api 2025-11-10 18:35:23,951 - DEBUG - Project uuid in Contrail is 5a1983f2-77f7-4008-9b59-3e79d7490f44 2025-11-10 18:35:24,090 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NamespaceDatabaseList?namespace_uuid=7e841245-3159-4293-8195-06d420ee9acf 2025-11-10 18:35:24,094 - INFO - Namespace ctest-new-default-69240602 with uuid 7e841245-3159-4293-8195-06d420ee9acf found in kube manager 2025-11-10 18:35:24,094 - INFO - Namespace ctest-new-default-69240602 verification passed 2025-11-10 18:35:24,333 - INFO - Project default-domain:k8s-ctest-non-default-59661362 for Namespace ctest-non-default-59661362 is seen in contrail-api 2025-11-10 18:35:24,333 - DEBUG - Project uuid in Contrail is eecb29dd-5efc-4b21-ab6e-3fa07dbed06c 2025-11-10 18:35:24,333 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NamespaceDatabaseList?namespace_uuid=7273cd45-af7d-4bb9-8d93-ffdae15943b8 2025-11-10 18:35:24,342 - INFO - Namespace ctest-non-default-59661362 with uuid 7273cd45-af7d-4bb9-8d93-ffdae15943b8 found in kube manager 2025-11-10 18:35:24,342 - INFO - Namespace ctest-non-default-59661362 verification passed 2025-11-10 18:35:24,348 - DEBUG - Pod ctest-pod-22080227 not in running state.Currently in Pending 2025-11-10 18:35:29,359 - DEBUG - Pod ctest-pod-22080227 not in running state.Currently in Pending 2025-11-10 18:35:34,371 - DEBUG - Pod ctest-pod-22080227 not in running state.Currently in Pending 2025-11-10 18:35:39,397 - INFO - Pod ctest-pod-22080227 is in running state.Got IP 10.47.255.251 2025-11-10 18:35:39,453 - DEBUG - Pod ctest-pod-22080227 has vmi 05d2eece-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:39,453 - INFO - Verified pod ctest-pod-22080227 in contrail-api 2025-11-10 18:35:39,468 - DEBUG - VMI 05d2eece-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:39,469 - INFO - Verified Pod ctest-pod-22080227 in agent 10.0.0.129 2025-11-10 18:35:39,469 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=4daac3bf-f63d-4cef-bc21-358e47551a58 2025-11-10 18:35:39,474 - INFO - Pod ctest-pod-22080227 with uuid 4daac3bf-f63d-4cef-bc21-358e47551a58 found in kube manager 2025-11-10 18:35:39,474 - INFO - Pod ctest-pod-22080227 verification passed 2025-11-10 18:35:39,482 - INFO - Pod ctest-pod-30809959 is in running state.Got IP 10.47.255.250 2025-11-10 18:35:39,496 - DEBUG - Pod ctest-pod-30809959 has vmi 06558df2-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:39,496 - INFO - Verified pod ctest-pod-30809959 in contrail-api 2025-11-10 18:35:39,509 - DEBUG - VMI 06558df2-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:39,509 - INFO - Verified Pod ctest-pod-30809959 in agent 10.0.0.129 2025-11-10 18:35:39,509 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=42be9da6-2e23-4c07-b722-c28c5824c31e 2025-11-10 18:35:39,514 - INFO - Pod ctest-pod-30809959 with uuid 42be9da6-2e23-4c07-b722-c28c5824c31e found in kube manager 2025-11-10 18:35:39,514 - INFO - Pod ctest-pod-30809959 verification passed 2025-11-10 18:35:39,523 - INFO - Pod ctest-pod-71702726 is in running state.Got IP 10.47.255.249 2025-11-10 18:35:39,585 - DEBUG - Pod ctest-pod-71702726 has vmi 06f500bc-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:39,585 - INFO - Verified pod ctest-pod-71702726 in contrail-api 2025-11-10 18:35:39,599 - DEBUG - VMI 06f500bc-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:39,599 - INFO - Verified Pod ctest-pod-71702726 in agent 10.0.0.129 2025-11-10 18:35:39,599 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=37ac037f-22dc-48ad-9872-55ff44d6982e 2025-11-10 18:35:39,603 - INFO - Pod ctest-pod-71702726 with uuid 37ac037f-22dc-48ad-9872-55ff44d6982e found in kube manager 2025-11-10 18:35:39,603 - INFO - Pod ctest-pod-71702726 verification passed 2025-11-10 18:35:39,610 - INFO - Pod ctest-pod-16506760 is in running state.Got IP 10.47.255.248 2025-11-10 18:35:39,665 - DEBUG - Pod ctest-pod-16506760 has vmi 076e8608-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:39,665 - INFO - Verified pod ctest-pod-16506760 in contrail-api 2025-11-10 18:35:39,679 - DEBUG - VMI 076e8608-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:39,680 - INFO - Verified Pod ctest-pod-16506760 in agent 10.0.0.129 2025-11-10 18:35:39,680 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=7f6add15-b79f-4d7f-89b4-38cccb627aca 2025-11-10 18:35:39,684 - INFO - Pod ctest-pod-16506760 with uuid 7f6add15-b79f-4d7f-89b4-38cccb627aca found in kube manager 2025-11-10 18:35:39,684 - INFO - Pod ctest-pod-16506760 verification passed 2025-11-10 18:35:39,693 - INFO - Pod ctest-pod-55044620 is in running state.Got IP 10.47.255.247 2025-11-10 18:35:39,707 - DEBUG - Pod ctest-pod-55044620 has vmi 07e6bf7e-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:39,707 - INFO - Verified pod ctest-pod-55044620 in contrail-api 2025-11-10 18:35:39,721 - DEBUG - VMI 07e6bf7e-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:39,721 - INFO - Verified Pod ctest-pod-55044620 in agent 10.0.0.129 2025-11-10 18:35:39,721 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=82310cfd-6a83-4b67-b81a-d0b7bfd7e59a 2025-11-10 18:35:39,725 - INFO - Pod ctest-pod-55044620 with uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a found in kube manager 2025-11-10 18:35:39,725 - INFO - Pod ctest-pod-55044620 verification passed 2025-11-10 18:35:39,732 - DEBUG - Pod ctest-pod-27738480 not in running state.Currently in Pending 2025-11-10 18:35:44,740 - INFO - Pod ctest-pod-27738480 is in running state.Got IP 10.47.255.245 2025-11-10 18:35:44,793 - DEBUG - Pod ctest-pod-27738480 has vmi 08dd1ffe-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:44,793 - INFO - Verified pod ctest-pod-27738480 in contrail-api 2025-11-10 18:35:44,807 - DEBUG - VMI 08dd1ffe-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:44,807 - INFO - Verified Pod ctest-pod-27738480 in agent 10.0.0.129 2025-11-10 18:35:44,807 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=0d84ce4a-7d70-4441-af0e-0681403d41cb 2025-11-10 18:35:44,812 - INFO - Pod ctest-pod-27738480 with uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb found in kube manager 2025-11-10 18:35:44,813 - INFO - Pod ctest-pod-27738480 verification passed 2025-11-10 18:35:44,821 - INFO - Pod ctest-pod-50111810 is in running state.Got IP 10.47.255.246 2025-11-10 18:35:44,836 - DEBUG - Pod ctest-pod-50111810 has vmi 0869dd50-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:44,836 - INFO - Verified pod ctest-pod-50111810 in contrail-api 2025-11-10 18:35:44,847 - DEBUG - VMI 0869dd50-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:44,848 - INFO - Verified Pod ctest-pod-50111810 in agent 10.0.0.129 2025-11-10 18:35:44,848 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 2025-11-10 18:35:44,852 - INFO - Pod ctest-pod-50111810 with uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 found in kube manager 2025-11-10 18:35:44,852 - INFO - Pod ctest-pod-50111810 verification passed 2025-11-10 18:35:44,859 - INFO - Pod ctest-pod-68851038 is in running state.Got IP 10.47.255.244 2025-11-10 18:35:44,915 - DEBUG - Pod ctest-pod-68851038 has vmi 09494e72-be64-11f0-83b4-fa163efd64b5 2025-11-10 18:35:44,915 - INFO - Verified pod ctest-pod-68851038 in contrail-api 2025-11-10 18:35:44,927 - DEBUG - VMI 09494e72-be64-11f0-83b4-fa163efd64b5 is active in agent 10.0.0.129 2025-11-10 18:35:44,927 - INFO - Verified Pod ctest-pod-68851038 in agent 10.0.0.129 2025-11-10 18:35:44,927 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=8d75cdcd-fc54-4344-88fb-5f238d79a46e 2025-11-10 18:35:44,932 - INFO - Pod ctest-pod-68851038 with uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e found in kube manager 2025-11-10 18:35:44,933 - INFO - Pod ctest-pod-68851038 verification passed 2025-11-10 18:35:44,933 - INFO - ================================================================================ 2025-11-10 18:35:44,934 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2025-11-10 18:35:44,934 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2025-11-10 18:35:45,144 - DEBUG - Nothing to compare xmpp stats {'10.0.0.129': {'10.0.0.33': '0'}} with 2025-11-10 18:35:45,144 - INFO - Initial checks done. Running the testcase now 2025-11-10 18:35:45,144 - INFO - 2025-11-10 18:35:45,200 - DEBUG - [Pod ctest-pod-22080227] Cmd: echo ctest-pod-22080227 > /usr/share/nginx/html/index.html, Output: 2025-11-10 18:35:45,262 - DEBUG - [Pod ctest-pod-30809959] Cmd: echo ctest-pod-30809959 > /usr/share/nginx/html/index.html, Output: 2025-11-10 18:35:45,319 - DEBUG - [Pod ctest-pod-71702726] Cmd: echo ctest-pod-71702726 > /usr/share/nginx/html/index.html, Output: 2025-11-10 18:35:45,383 - DEBUG - [Pod ctest-pod-16506760] Cmd: echo ctest-pod-16506760 > /usr/share/nginx/html/index.html, Output: 2025-11-10 18:35:45,387 - DEBUG - Service ctest-nginx-svc-96375746 not present 2025-11-10 18:35:45,387 - INFO - Creating service ctest-nginx-svc-96375746 2025-11-10 18:35:45,401 - INFO - Created Service ctest-nginx-svc-96375746 2025-11-10 18:35:45,415 - WARNING - Service UUID dc114de0-bc9f-493c-8665-bc2a052a3433 not yet found in contrail-api 2025-11-10 18:35:46,433 - INFO - Validated that Service ctest-nginx-svc-96375746 is seen in contrail-api 2025-11-10 18:35:46,433 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_LoadbalancerDatabaseList?x=dc114de0-bc9f-493c-8665-bc2a052a3433 2025-11-10 18:35:46,444 - INFO - Service ctest-nginx-svc-96375746 with uuid dc114de0-bc9f-493c-8665-bc2a052a3433 found in kube manager 2025-11-10 18:35:46,445 - INFO - Service ctest-nginx-svc-96375746 verification passed 2025-11-10 18:35:47,515 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:35:47,515 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:35:47,515 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:35:47,573 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:35:47,573 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:35:47,573 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:35:47,624 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:35:47,624 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:35:47,624 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:35:47,685 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-22080227 2025-11-10 18:35:47,685 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:35:47,685 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-22080227 2025-11-10 18:35:47,686 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-22080227': 1, 'ctest-pod-30809959': 3} 2025-11-10 18:35:47,747 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:35:47,747 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:35:47,747 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:35:47,797 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:35:47,797 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:35:47,797 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:35:47,797 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-22080227': 1, 'ctest-pod-30809959': 1} 2025-11-10 18:35:47,802 - DEBUG - Network policy ingress-on-service not present 2025-11-10 18:35:47,802 - INFO - Creating Network Policy ingress-on-service 2025-11-10 18:35:47,944 - INFO - Network policy found in k8s 2025-11-10 18:35:47,944 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=2fe5472b-4342-4729-ba67-c4073f495566 2025-11-10 18:35:47,950 - INFO - Network Policy ingress-on-service with uuid 2fe5472b-4342-4729-ba67-c4073f495566 found in kube manager 2025-11-10 18:35:48,007 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2025-11-10 18:35:48,008 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2025-11-10 18:35:48,008 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=2fe5472b-4342-4729-ba67-c4073f495566 2025-11-10 18:35:48,035 - WARNING - Network policy with name ingress-on-service not found in agent 2025-11-10 18:35:50,036 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=2fe5472b-4342-4729-ba67-c4073f495566 2025-11-10 18:35:50,063 - INFO - Network policy with name ingress-on-service found in agent 2025-11-10 18:35:50,064 - INFO - Network Policy ingress-on-service verification passed 2025-11-10 18:35:55,111 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:35:55,111 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:35:55,111 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:00,170 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:00,171 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:00,171 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:05,223 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:05,223 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:05,223 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:10,275 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:10,275 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:10,275 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:15,331 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:15,331 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:15,331 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:20,390 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:20,390 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:20,391 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:25,445 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:25,445 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:25,445 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:30,512 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:30,512 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:30,512 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:35,566 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:35,567 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:35,567 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:40,627 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:40,627 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:40,627 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:40,627 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-22080227': 0, 'ctest-pod-30809959': 0} 2025-11-10 18:36:40,707 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,707 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:40,707 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,776 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,776 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:40,776 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,828 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,828 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:40,828 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,869 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:36:40,869 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:40,869 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:36:40,933 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,933 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:40,933 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:40,987 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:36:40,987 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:40,987 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:36:41,041 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:41,041 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:36:41,041 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:36:41,041 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-22080227': 1, 'ctest-pod-30809959': 6} 2025-11-10 18:36:46,092 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:46,092 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:46,092 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:51,148 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:51,149 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:51,149 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:56,205 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:56,205 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:36:56,205 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:01,276 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:01,276 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:01,276 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:06,339 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:06,340 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:06,340 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:11,397 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:11,397 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:11,397 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:16,455 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:16,455 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:16,455 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:21,502 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:21,502 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:21,502 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:26,559 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:26,559 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:26,559 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:28,808 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2025-11-10 18:37:38,879 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:38,879 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:38,879 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:37:38,879 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-22080227': 0, 'ctest-pod-30809959': 0} 2025-11-10 18:37:40,935 - DEBUG - [Pod ctest-pod-50111810] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.783 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.087 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.088 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.087/0.319/0.783 ms 2025-11-10 18:37:40,935 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-50111810 with expectation True passed 2025-11-10 18:37:42,986 - DEBUG - [Pod ctest-pod-55044620] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.875 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.086 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.105 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.086/0.355/0.875 ms 2025-11-10 18:37:42,987 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-55044620 with expectation True passed 2025-11-10 18:37:45,039 - DEBUG - [Pod ctest-pod-27738480] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.751 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.083 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.101 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.083/0.311/0.751 ms 2025-11-10 18:37:45,039 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-27738480 with expectation True passed 2025-11-10 18:37:45,096 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:37:45,096 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-11-10 18:37:45,096 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:37:45,096 - INFO - wget check of of http://10.47.255.251 from ctest-pod-50111810 passed 2025-11-10 18:37:45,155 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:37:45,155 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-11-10 18:37:45,155 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:37:45,155 - INFO - wget check of of http://10.47.255.250 from ctest-pod-50111810 passed 2025-11-10 18:37:50,214 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-11-10 18:37:50,214 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-11-10 18:37:50,214 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-11-10 18:37:50,214 - INFO - wget check of of http://10.47.255.250 from ctest-pod-55044620 passed 2025-11-10 18:37:55,265 - DEBUG - [Pod ctest-pod-68851038] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-11-10 18:37:55,265 - DEBUG - [Pod Pod ctest-pod-68851038] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-11-10 18:37:55,265 - DEBUG - [Pod Pod ctest-pod-68851038] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-11-10 18:37:55,265 - INFO - wget check of of http://10.47.255.251 from ctest-pod-68851038 passed 2025-11-10 18:37:55,275 - INFO - Updating Network Policy ingress-on-service 2025-11-10 18:38:00,346 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:00,346 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:00,346 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:05,399 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:05,400 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:05,400 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:10,451 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:10,451 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:10,451 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:15,512 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:15,513 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:15,513 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:20,570 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:20,570 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:20,570 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:25,620 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:25,620 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:25,620 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:30,678 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:30,678 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:30,678 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:35,734 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:35,734 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:35,734 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:40,791 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:40,791 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:40,791 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:45,848 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:45,848 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:45,848 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) wget: download timed out 2025-11-10 18:38:45,848 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-22080227': 0, 'ctest-pod-30809959': 0} 2025-11-10 18:38:45,917 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:45,917 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:38:45,917 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:45,972 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:38:45,972 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:38:45,972 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30809959 2025-11-10 18:38:45,972 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-22080227': 1, 'ctest-pod-30809959': 1} 2025-11-10 18:38:46,031 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:46,031 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:38:46,031 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:46,107 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:46,108 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:38:46,108 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: ctest-pod-30809959 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:46,160 - DEBUG - [Pod ctest-pod-27738480] Cmd: wget http://10.99.253.88:80/ -O - -T 5 -t 1 , Output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:46,161 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd wget http://10.99.253.88:80/ -O - -T 5 -t 1 passed 2025-11-10 18:38:46,161 - DEBUG - [Pod Pod ctest-pod-27738480] Cmd output: ctest-pod-22080227 Connecting to 10.99.253.88:80 (10.99.253.88:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-11-10 18:38:46,161 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-22080227': 1, 'ctest-pod-30809959': 2} 2025-11-10 18:38:48,211 - DEBUG - [Pod ctest-pod-50111810] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.454 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.085 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.100 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.085/0.213/0.454 ms 2025-11-10 18:38:48,211 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-50111810 with expectation True passed 2025-11-10 18:38:50,260 - DEBUG - [Pod ctest-pod-55044620] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.442 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.083 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.108 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.083/0.211/0.442 ms 2025-11-10 18:38:50,260 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-55044620 with expectation True passed 2025-11-10 18:38:52,315 - DEBUG - [Pod ctest-pod-27738480] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.720 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.087 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.121 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.087/0.309/0.720 ms 2025-11-10 18:38:52,315 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-27738480 with expectation True passed 2025-11-10 18:38:52,379 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:38:52,380 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-11-10 18:38:52,380 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:38:52,380 - INFO - wget check of of http://10.47.255.251 from ctest-pod-50111810 passed 2025-11-10 18:38:52,440 - DEBUG - [Pod ctest-pod-50111810] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:38:52,440 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-11-10 18:38:52,441 - DEBUG - [Pod Pod ctest-pod-50111810] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:38:52,441 - INFO - wget check of of http://10.47.255.250 from ctest-pod-50111810 passed 2025-11-10 18:38:57,494 - DEBUG - [Pod ctest-pod-55044620] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-11-10 18:38:57,495 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-11-10 18:38:57,495 - DEBUG - [Pod Pod ctest-pod-55044620] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-11-10 18:38:57,495 - INFO - wget check of of http://10.47.255.250 from ctest-pod-55044620 passed 2025-11-10 18:38:57,556 - DEBUG - [Pod ctest-pod-68851038] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:38:57,557 - DEBUG - [Pod Pod ctest-pod-68851038] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-11-10 18:38:57,557 - DEBUG - [Pod Pod ctest-pod-68851038] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-11-10 18:38:57,557 - INFO - wget check of of http://10.47.255.251 from ctest-pod-68851038 passed 2025-11-10 18:38:57,557 - INFO - Deleting Network Policy : ingress-on-service 2025-11-10 18:38:57,558 - INFO - Deleting service : ctest-nginx-svc-96375746 2025-11-10 18:38:57,851 - DEBUG - No XMPP flaps were noticed during the test 2025-11-10 18:38:57,851 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:13] 2025-11-10 18:38:57,852 - INFO - -------------------------------------------------------------------------------- 2025-11-10 18:38:57,856 - INFO - Deleting pod ctest-new-default-69240602:ctest-pod-22080227 2025-11-10 18:38:57,856 - INFO - Deleting pod ctest-new-default-69240602:ctest-pod-30809959 2025-11-10 18:38:57,857 - INFO - Deleting pod ctest-non-default-59661362:ctest-pod-71702726 2025-11-10 18:38:57,857 - INFO - Deleting pod ctest-non-default-59661362:ctest-pod-16506760 2025-11-10 18:38:57,858 - INFO - Deleting pod ctest-new-default-69240602:ctest-pod-55044620 2025-11-10 18:38:57,859 - INFO - Deleting pod ctest-new-default-69240602:ctest-pod-50111810 2025-11-10 18:38:57,859 - INFO - Deleting pod ctest-non-default-59661362:ctest-pod-27738480 2025-11-10 18:38:57,860 - INFO - Deleting pod ctest-non-default-59661362:ctest-pod-68851038 2025-11-10 18:38:57,998 - WARNING - Pod uuid 4daac3bf-f63d-4cef-bc21-358e47551a58 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:57,999 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:57,999 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:58,011 - WARNING - Pod uuid 7f6add15-b79f-4d7f-89b4-38cccb627aca is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:58,012 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:58,013 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:58,014 - WARNING - Pod uuid 37ac037f-22dc-48ad-9872-55ff44d6982e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:38:58,015 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:03,011 - DEBUG - Pod 4daac3bf-f63d-4cef-bc21-358e47551a58 is not in agent 10.0.0.129 VM list 2025-11-10 18:39:03,018 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:03,018 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:03,037 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:03,038 - DEBUG - VMI 05d2eece-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:03,038 - INFO - Verified that pod ctest-pod-22080227 is removed in agent 2025-11-10 18:39:03,039 - DEBUG - Pod 37ac037f-22dc-48ad-9872-55ff44d6982e is not in agent 10.0.0.129 VM list 2025-11-10 18:39:03,040 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:03,040 - DEBUG - Pod 7f6add15-b79f-4d7f-89b4-38cccb627aca is not in agent 10.0.0.129 VM list 2025-11-10 18:39:03,042 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:03,050 - DEBUG - VMI 06f500bc-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:03,050 - INFO - Verified that pod ctest-pod-71702726 is removed in agent 2025-11-10 18:39:03,051 - DEBUG - VMI 076e8608-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:03,051 - INFO - Verified that pod ctest-pod-16506760 is removed in agent 2025-11-10 18:39:08,035 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:08,038 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:08,054 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:08,056 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:08,056 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:13,044 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:13,046 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:13,065 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:13,066 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:13,067 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:18,055 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:18,056 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:18,080 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:18,081 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:18,081 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:23,067 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:23,070 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:23,095 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:23,097 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:23,097 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:28,078 - WARNING - Pod uuid 8d75cdcd-fc54-4344-88fb-5f238d79a46e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:28,079 - WARNING - Pod uuid bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:28,108 - WARNING - Pod uuid 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:28,110 - WARNING - Pod uuid 0d84ce4a-7d70-4441-af0e-0681403d41cb is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:28,111 - WARNING - Pod uuid 42be9da6-2e23-4c07-b722-c28c5824c31e is still seen in agent 10.0.0.129 VM list 2025-11-10 18:39:33,090 - DEBUG - Pod 8d75cdcd-fc54-4344-88fb-5f238d79a46e is not in agent 10.0.0.129 VM list 2025-11-10 18:39:33,093 - DEBUG - Pod bcda73b8-2ed8-47a1-bd8a-d0f5209bc120 is not in agent 10.0.0.129 VM list 2025-11-10 18:39:33,101 - DEBUG - VMI 09494e72-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:33,101 - INFO - Verified that pod ctest-pod-68851038 is removed in agent 2025-11-10 18:39:33,102 - DEBUG - VMI 0869dd50-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:33,103 - INFO - Verified that pod ctest-pod-50111810 is removed in agent 2025-11-10 18:39:33,121 - DEBUG - Pod 82310cfd-6a83-4b67-b81a-d0b7bfd7e59a is not in agent 10.0.0.129 VM list 2025-11-10 18:39:33,124 - DEBUG - Pod 42be9da6-2e23-4c07-b722-c28c5824c31e is not in agent 10.0.0.129 VM list 2025-11-10 18:39:33,125 - DEBUG - Pod 0d84ce4a-7d70-4441-af0e-0681403d41cb is not in agent 10.0.0.129 VM list 2025-11-10 18:39:33,134 - DEBUG - VMI 07e6bf7e-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:33,134 - INFO - Verified that pod ctest-pod-55044620 is removed in agent 2025-11-10 18:39:33,135 - DEBUG - VMI 06558df2-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:33,135 - INFO - Verified that pod ctest-pod-30809959 is removed in agent 2025-11-10 18:39:33,136 - DEBUG - VMI 08dd1ffe-be64-11f0-83b4-fa163efd64b5 is removed from agent 10.0.0.129 2025-11-10 18:39:33,136 - INFO - Verified that pod ctest-pod-27738480 is removed in agent 2025-11-10 18:39:33,136 - INFO - Deleting namespace ctest-non-default-59661362 2025-11-10 18:39:33,146 - DEBUG - Namespace ctest-non-default-59661362 still in k8s 2025-11-10 18:39:35,153 - DEBUG - Namespace ctest-non-default-59661362 still in k8s 2025-11-10 18:39:37,160 - DEBUG - Namespace ctest-non-default-59661362 still in k8s 2025-11-10 18:39:39,165 - DEBUG - Namespace ctest-non-default-59661362 is not in k8s 2025-11-10 18:39:39,165 - DEBUG - No need to check project deletion in contrail-api 2025-11-10 18:39:39,165 - INFO - Deleting namespace ctest-new-default-69240602 2025-11-10 18:39:39,174 - DEBUG - Namespace ctest-new-default-69240602 still in k8s 2025-11-10 18:39:41,178 - DEBUG - Namespace ctest-new-default-69240602 still in k8s 2025-11-10 18:39:43,183 - DEBUG - Namespace ctest-new-default-69240602 still in k8s 2025-11-10 18:39:45,189 - DEBUG - Namespace ctest-new-default-69240602 is not in k8s 2025-11-10 18:39:45,189 - DEBUG - No need to check project deletion in contrail-api