2025-08-12 01:36:35,376 - INFO - Using existing project ['default-domain', 'k8s-default'](04f09fb4-3420-448a-b645-e1ec2b88c8ee) 2025-08-12 01:36:35,394 - DEBUG - Namespace ctest-new-default-98136278 not present 2025-08-12 01:36:35,400 - INFO - Created namespace ctest-new-default-98136278 2025-08-12 01:36:38,403 - DEBUG - Namespace ctest-non-default-47249989 not present 2025-08-12 01:36:38,415 - INFO - Created namespace ctest-non-default-47249989 2025-08-12 01:36:41,433 - DEBUG - Pod ctest-pod-83698808 not present 2025-08-12 01:36:41,433 - INFO - Creating Pod ctest-pod-83698808 2025-08-12 01:36:41,452 - DEBUG - Node for Pod ctest-pod-83698808 not yet populated 2025-08-12 01:36:44,459 - DEBUG - Pod : ctest-pod-83698808 UUID is 04bc01b2-cb63-4ea0-a98c-92245d66ad6c 2025-08-12 01:36:44,461 - DEBUG - Pod ctest-pod-03200971 not present 2025-08-12 01:36:44,461 - INFO - Creating Pod ctest-pod-03200971 2025-08-12 01:36:44,477 - DEBUG - Node for Pod ctest-pod-03200971 not yet populated 2025-08-12 01:36:47,482 - DEBUG - Pod : ctest-pod-03200971 UUID is 7bc7d613-c0d2-49ae-b9f4-574a98593467 2025-08-12 01:36:47,484 - DEBUG - Pod ctest-pod-44131503 not present 2025-08-12 01:36:47,485 - INFO - Creating Pod ctest-pod-44131503 2025-08-12 01:36:47,503 - DEBUG - Pod : ctest-pod-44131503 UUID is d2be86e9-a449-4f31-bc1b-09b3f7a220e6 2025-08-12 01:36:47,510 - DEBUG - Pod ctest-pod-19935220 not present 2025-08-12 01:36:47,510 - INFO - Creating Pod ctest-pod-19935220 2025-08-12 01:36:47,551 - DEBUG - Pod : ctest-pod-19935220 UUID is f373f0d8-050f-4f9b-877c-e49e7487fa7f 2025-08-12 01:36:47,554 - DEBUG - Pod ctest-pod-01324802 not present 2025-08-12 01:36:47,554 - INFO - Creating Pod ctest-pod-01324802 2025-08-12 01:36:47,574 - DEBUG - Pod : ctest-pod-01324802 UUID is 653172c6-541e-494b-98af-d7e6b171d84b 2025-08-12 01:36:47,578 - DEBUG - Pod ctest-pod-13192457 not present 2025-08-12 01:36:47,578 - INFO - Creating Pod ctest-pod-13192457 2025-08-12 01:36:47,592 - DEBUG - Pod : ctest-pod-13192457 UUID is be440c65-a1e7-420b-8b81-e8b029bec0b2 2025-08-12 01:36:47,599 - DEBUG - Pod ctest-pod-45836418 not present 2025-08-12 01:36:47,600 - INFO - Creating Pod ctest-pod-45836418 2025-08-12 01:36:47,619 - DEBUG - Pod : ctest-pod-45836418 UUID is eaba0f12-c404-4814-a60c-1b05de6e6e7c 2025-08-12 01:36:47,621 - DEBUG - Pod ctest-pod-91762201 not present 2025-08-12 01:36:47,621 - INFO - Creating Pod ctest-pod-91762201 2025-08-12 01:36:47,639 - DEBUG - Pod : ctest-pod-91762201 UUID is 9a1207c0-0545-4e86-8186-4529668381ca 2025-08-12 01:36:47,799 - INFO - Project default-domain:k8s-ctest-new-default-98136278 for Namespace ctest-new-default-98136278 is seen in contrail-api 2025-08-12 01:36:47,799 - DEBUG - Project uuid in Contrail is fe01955f-59b8-4260-aaa1-fe7ac548e4d6 2025-08-12 01:36:47,993 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_NamespaceDatabaseList?namespace_uuid=2334bfd4-e499-48de-a468-4852a118a433 2025-08-12 01:36:48,020 - INFO - Namespace ctest-new-default-98136278 with uuid 2334bfd4-e499-48de-a468-4852a118a433 found in kube manager 2025-08-12 01:36:48,020 - INFO - Namespace ctest-new-default-98136278 verification passed 2025-08-12 01:36:48,211 - INFO - Project default-domain:k8s-ctest-non-default-47249989 for Namespace ctest-non-default-47249989 is seen in contrail-api 2025-08-12 01:36:48,212 - DEBUG - Project uuid in Contrail is 52efda9b-606e-40c7-98b3-d9e4bc1b00f3 2025-08-12 01:36:48,212 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_NamespaceDatabaseList?namespace_uuid=08b7764f-12a1-4af4-ad0a-b981fff7417a 2025-08-12 01:36:48,216 - INFO - Namespace ctest-non-default-47249989 with uuid 08b7764f-12a1-4af4-ad0a-b981fff7417a found in kube manager 2025-08-12 01:36:48,216 - INFO - Namespace ctest-non-default-47249989 verification passed 2025-08-12 01:36:48,221 - DEBUG - Pod ctest-pod-83698808 not in running state.Currently in Pending 2025-08-12 01:36:53,227 - DEBUG - Pod ctest-pod-83698808 not in running state.Currently in Pending 2025-08-12 01:36:58,233 - INFO - Pod ctest-pod-83698808 is in running state.Got IP 10.47.255.251 2025-08-12 01:36:58,284 - DEBUG - Pod ctest-pod-83698808 has vmi ccdd262c-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:36:58,284 - INFO - Verified pod ctest-pod-83698808 in contrail-api 2025-08-12 01:36:58,296 - DEBUG - VMI ccdd262c-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:36:58,297 - INFO - Verified Pod ctest-pod-83698808 in agent 10.0.0.249 2025-08-12 01:36:58,297 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=04bc01b2-cb63-4ea0-a98c-92245d66ad6c 2025-08-12 01:36:58,301 - INFO - Pod ctest-pod-83698808 with uuid 04bc01b2-cb63-4ea0-a98c-92245d66ad6c found in kube manager 2025-08-12 01:36:58,301 - INFO - Pod ctest-pod-83698808 verification passed 2025-08-12 01:36:58,306 - INFO - Pod ctest-pod-03200971 is in running state.Got IP 10.47.255.250 2025-08-12 01:36:58,318 - DEBUG - Pod ctest-pod-03200971 has vmi cdf2b1b2-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:36:58,319 - INFO - Verified pod ctest-pod-03200971 in contrail-api 2025-08-12 01:36:58,329 - DEBUG - VMI cdf2b1b2-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:36:58,329 - INFO - Verified Pod ctest-pod-03200971 in agent 10.0.0.249 2025-08-12 01:36:58,329 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=7bc7d613-c0d2-49ae-b9f4-574a98593467 2025-08-12 01:36:58,333 - INFO - Pod ctest-pod-03200971 with uuid 7bc7d613-c0d2-49ae-b9f4-574a98593467 found in kube manager 2025-08-12 01:36:58,333 - INFO - Pod ctest-pod-03200971 verification passed 2025-08-12 01:36:58,338 - INFO - Pod ctest-pod-44131503 is in running state.Got IP 10.47.255.249 2025-08-12 01:36:58,391 - DEBUG - Pod ctest-pod-44131503 has vmi cff578be-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:36:58,391 - INFO - Verified pod ctest-pod-44131503 in contrail-api 2025-08-12 01:36:58,403 - DEBUG - VMI cff578be-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:36:58,403 - INFO - Verified Pod ctest-pod-44131503 in agent 10.0.0.249 2025-08-12 01:36:58,403 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=d2be86e9-a449-4f31-bc1b-09b3f7a220e6 2025-08-12 01:36:58,407 - INFO - Pod ctest-pod-44131503 with uuid d2be86e9-a449-4f31-bc1b-09b3f7a220e6 found in kube manager 2025-08-12 01:36:58,407 - INFO - Pod ctest-pod-44131503 verification passed 2025-08-12 01:36:58,412 - DEBUG - Pod ctest-pod-19935220 not in running state.Currently in Pending 2025-08-12 01:37:03,418 - INFO - Pod ctest-pod-19935220 is in running state.Got IP 10.47.255.248 2025-08-12 01:37:03,430 - DEBUG - Pod ctest-pod-19935220 has vmi d065eeb4-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:37:03,430 - INFO - Verified pod ctest-pod-19935220 in contrail-api 2025-08-12 01:37:03,441 - DEBUG - VMI d065eeb4-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:37:03,441 - INFO - Verified Pod ctest-pod-19935220 in agent 10.0.0.249 2025-08-12 01:37:03,442 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=f373f0d8-050f-4f9b-877c-e49e7487fa7f 2025-08-12 01:37:03,445 - INFO - Pod ctest-pod-19935220 with uuid f373f0d8-050f-4f9b-877c-e49e7487fa7f found in kube manager 2025-08-12 01:37:03,446 - INFO - Pod ctest-pod-19935220 verification passed 2025-08-12 01:37:03,450 - INFO - Pod ctest-pod-01324802 is in running state.Got IP 10.47.255.247 2025-08-12 01:37:03,505 - DEBUG - Pod ctest-pod-01324802 has vmi d0e4db16-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:37:03,505 - INFO - Verified pod ctest-pod-01324802 in contrail-api 2025-08-12 01:37:03,518 - DEBUG - VMI d0e4db16-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:37:03,519 - INFO - Verified Pod ctest-pod-01324802 in agent 10.0.0.249 2025-08-12 01:37:03,519 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=653172c6-541e-494b-98af-d7e6b171d84b 2025-08-12 01:37:03,523 - INFO - Pod ctest-pod-01324802 with uuid 653172c6-541e-494b-98af-d7e6b171d84b found in kube manager 2025-08-12 01:37:03,523 - INFO - Pod ctest-pod-01324802 verification passed 2025-08-12 01:37:03,528 - INFO - Pod ctest-pod-45836418 is in running state.Got IP 10.47.255.245 2025-08-12 01:37:03,580 - DEBUG - Pod ctest-pod-45836418 has vmi d1daf88e-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:37:03,580 - INFO - Verified pod ctest-pod-45836418 in contrail-api 2025-08-12 01:37:03,592 - DEBUG - VMI d1daf88e-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:37:03,592 - INFO - Verified Pod ctest-pod-45836418 in agent 10.0.0.249 2025-08-12 01:37:03,592 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=eaba0f12-c404-4814-a60c-1b05de6e6e7c 2025-08-12 01:37:03,596 - INFO - Pod ctest-pod-45836418 with uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c found in kube manager 2025-08-12 01:37:03,596 - INFO - Pod ctest-pod-45836418 verification passed 2025-08-12 01:37:03,601 - INFO - Pod ctest-pod-13192457 is in running state.Got IP 10.47.255.246 2025-08-12 01:37:03,616 - DEBUG - Pod ctest-pod-13192457 has vmi d15669a2-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:37:03,617 - INFO - Verified pod ctest-pod-13192457 in contrail-api 2025-08-12 01:37:03,627 - DEBUG - VMI d15669a2-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:37:03,627 - INFO - Verified Pod ctest-pod-13192457 in agent 10.0.0.249 2025-08-12 01:37:03,627 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=be440c65-a1e7-420b-8b81-e8b029bec0b2 2025-08-12 01:37:03,631 - INFO - Pod ctest-pod-13192457 with uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 found in kube manager 2025-08-12 01:37:03,631 - INFO - Pod ctest-pod-13192457 verification passed 2025-08-12 01:37:03,636 - INFO - Pod ctest-pod-91762201 is in running state.Got IP 10.47.255.244 2025-08-12 01:37:03,692 - DEBUG - Pod ctest-pod-91762201 has vmi d247c6da-771c-11f0-bea2-fa163e5be5c4 2025-08-12 01:37:03,692 - INFO - Verified pod ctest-pod-91762201 in contrail-api 2025-08-12 01:37:03,704 - DEBUG - VMI d247c6da-771c-11f0-bea2-fa163e5be5c4 is active in agent 10.0.0.249 2025-08-12 01:37:03,704 - INFO - Verified Pod ctest-pod-91762201 in agent 10.0.0.249 2025-08-12 01:37:03,704 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_VirtualMachineDatabaseList?x=9a1207c0-0545-4e86-8186-4529668381ca 2025-08-12 01:37:03,708 - INFO - Pod ctest-pod-91762201 with uuid 9a1207c0-0545-4e86-8186-4529668381ca found in kube manager 2025-08-12 01:37:03,708 - INFO - Pod ctest-pod-91762201 verification passed 2025-08-12 01:37:03,708 - INFO - ================================================================================ 2025-08-12 01:37:03,708 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2025-08-12 01:37:03,708 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2025-08-12 01:37:03,927 - DEBUG - Nothing to compare xmpp stats {'10.0.0.249': {'10.0.0.38': '0'}} with 2025-08-12 01:37:03,927 - INFO - Initial checks done. Running the testcase now 2025-08-12 01:37:03,927 - INFO - 2025-08-12 01:37:03,978 - DEBUG - [Pod ctest-pod-83698808] Cmd: echo ctest-pod-83698808 > /usr/share/nginx/html/index.html, Output: 2025-08-12 01:37:04,038 - DEBUG - [Pod ctest-pod-03200971] Cmd: echo ctest-pod-03200971 > /usr/share/nginx/html/index.html, Output: 2025-08-12 01:37:04,092 - DEBUG - [Pod ctest-pod-44131503] Cmd: echo ctest-pod-44131503 > /usr/share/nginx/html/index.html, Output: 2025-08-12 01:37:04,150 - DEBUG - [Pod ctest-pod-19935220] Cmd: echo ctest-pod-19935220 > /usr/share/nginx/html/index.html, Output: 2025-08-12 01:37:04,153 - DEBUG - Service ctest-nginx-svc-76825044 not present 2025-08-12 01:37:04,153 - INFO - Creating service ctest-nginx-svc-76825044 2025-08-12 01:37:04,165 - INFO - Created Service ctest-nginx-svc-76825044 2025-08-12 01:37:04,170 - WARNING - Service UUID 5c9b9cd8-cd1b-4835-b286-f4ea8d80e05f not yet found in contrail-api 2025-08-12 01:37:05,181 - INFO - Validated that Service ctest-nginx-svc-76825044 is seen in contrail-api 2025-08-12 01:37:05,181 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_LoadbalancerDatabaseList?x=5c9b9cd8-cd1b-4835-b286-f4ea8d80e05f 2025-08-12 01:37:05,185 - INFO - Service ctest-nginx-svc-76825044 with uuid 5c9b9cd8-cd1b-4835-b286-f4ea8d80e05f found in kube manager 2025-08-12 01:37:05,186 - INFO - Service ctest-nginx-svc-76825044 verification passed 2025-08-12 01:37:06,239 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) ctest-pod-03200971 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,239 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:06,239 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) ctest-pod-03200971 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,284 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,284 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:06,284 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,353 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,353 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:06,353 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,354 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-83698808': 1, 'ctest-pod-03200971': 2} 2025-08-12 01:37:06,408 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,408 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:06,408 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,457 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,457 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:06,457 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:06,457 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-83698808': 1, 'ctest-pod-03200971': 1} 2025-08-12 01:37:06,460 - DEBUG - Network policy ingress-on-service not present 2025-08-12 01:37:06,461 - INFO - Creating Network Policy ingress-on-service 2025-08-12 01:37:06,472 - INFO - Network policy found in k8s 2025-08-12 01:37:06,473 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=bb780525-7e0b-4b7e-a9ea-522ded6c6fb0 2025-08-12 01:37:06,480 - INFO - Network Policy ingress-on-service with uuid bb780525-7e0b-4b7e-a9ea-522ded6c6fb0 found in kube manager 2025-08-12 01:37:06,509 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2025-08-12 01:37:06,509 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2025-08-12 01:37:06,510 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=bb780525-7e0b-4b7e-a9ea-522ded6c6fb0 2025-08-12 01:37:06,535 - WARNING - Network policy with name ingress-on-service not found in agent 2025-08-12 01:37:08,537 - DEBUG - Requesting: http://10.0.0.38:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=bb780525-7e0b-4b7e-a9ea-522ded6c6fb0 2025-08-12 01:37:08,575 - INFO - Network policy with name ingress-on-service found in agent 2025-08-12 01:37:08,576 - INFO - Network Policy ingress-on-service verification passed 2025-08-12 01:37:13,636 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:13,636 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:13,636 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:18,702 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:18,702 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:18,702 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:23,750 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:23,750 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:23,750 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:28,799 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:28,799 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:28,799 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:33,847 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:33,848 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:33,848 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:38,922 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:38,922 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:38,922 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:43,978 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:43,978 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:43,978 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:49,022 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:49,022 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:49,022 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:54,072 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:54,072 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:54,072 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:59,122 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:59,122 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:59,122 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:37:59,122 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-83698808': 0, 'ctest-pod-03200971': 0} 2025-08-12 01:37:59,173 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,173 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,173 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,221 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,221 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,221 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,251 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-83698808 2025-08-12 01:37:59,252 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,252 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-83698808 2025-08-12 01:37:59,300 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,300 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,300 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,351 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,351 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,351 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,403 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,404 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,404 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,458 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,458 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:37:59,458 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:37:59,458 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-83698808': 6, 'ctest-pod-03200971': 1} 2025-08-12 01:38:04,499 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:04,499 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:04,499 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:09,546 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:09,546 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:09,546 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:14,594 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:14,594 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:14,595 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:19,638 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:19,638 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:19,638 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:24,686 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:24,686 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:24,686 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:29,734 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:29,734 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:29,734 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:34,778 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:34,778 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:34,778 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:39,822 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:39,822 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:39,822 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:42,804 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2025-08-12 01:38:52,855 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:52,855 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:52,855 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:57,902 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:57,903 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:57,903 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:38:57,903 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-83698808': 0, 'ctest-pod-03200971': 0} 2025-08-12 01:38:59,951 - DEBUG - [Pod ctest-pod-13192457] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.723 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.068 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.061 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.061/0.284/0.723 ms 2025-08-12 01:38:59,951 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-13192457 with expectation True passed 2025-08-12 01:39:01,999 - DEBUG - [Pod ctest-pod-01324802] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.901 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.064 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.097 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.064/0.687/1.901 ms 2025-08-12 01:39:02,000 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-01324802 with expectation True passed 2025-08-12 01:39:04,042 - DEBUG - [Pod ctest-pod-45836418] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.620 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.070 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.085 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.070/0.258/0.620 ms 2025-08-12 01:39:04,043 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-45836418 with expectation True passed 2025-08-12 01:39:04,093 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:39:04,093 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-08-12 01:39:04,093 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:39:04,093 - INFO - wget check of of http://10.47.255.251 from ctest-pod-13192457 passed 2025-08-12 01:39:04,126 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:39:04,127 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-08-12 01:39:04,127 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:39:04,127 - INFO - wget check of of http://10.47.255.250 from ctest-pod-13192457 passed 2025-08-12 01:39:09,183 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-08-12 01:39:09,183 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-08-12 01:39:09,183 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-08-12 01:39:09,183 - INFO - wget check of of http://10.47.255.250 from ctest-pod-01324802 passed 2025-08-12 01:39:14,228 - DEBUG - [Pod ctest-pod-91762201] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-08-12 01:39:14,228 - DEBUG - [Pod Pod ctest-pod-91762201] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-08-12 01:39:14,228 - DEBUG - [Pod Pod ctest-pod-91762201] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-08-12 01:39:14,228 - INFO - wget check of of http://10.47.255.251 from ctest-pod-91762201 passed 2025-08-12 01:39:14,235 - INFO - Updating Network Policy ingress-on-service 2025-08-12 01:39:19,290 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:19,290 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:19,291 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:24,334 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:24,334 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:24,334 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:29,382 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:29,382 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:29,382 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:34,426 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:34,426 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:34,426 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:39,471 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:39,471 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:39,471 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:44,515 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:44,515 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:44,515 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:49,563 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:49,564 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:49,564 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:54,611 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:54,611 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:54,611 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:59,655 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:59,655 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:39:59,655 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:40:04,702 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:40:04,702 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:40:04,702 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.111.202.120:80 (10.111.202.120:80) wget: download timed out 2025-08-12 01:40:04,702 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-83698808': 0, 'ctest-pod-03200971': 0} 2025-08-12 01:40:04,748 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,748 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:04,748 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,791 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,791 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:04,791 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,848 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,848 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:04,848 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,895 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,895 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:04,895 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,932 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,932 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:04,933 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,979 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,979 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:04,979 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:04,979 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-83698808': 5, 'ctest-pod-03200971': 1} 2025-08-12 01:40:05,028 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:05,028 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:05,028 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: ctest-pod-03200971 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:05,080 - DEBUG - [Pod ctest-pod-45836418] Cmd: wget http://10.111.202.120:80/ -O - -T 5 -t 1 , Output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:05,080 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd wget http://10.111.202.120:80/ -O - -T 5 -t 1 passed 2025-08-12 01:40:05,080 - DEBUG - [Pod Pod ctest-pod-45836418] Cmd output: ctest-pod-83698808 Connecting to 10.111.202.120:80 (10.111.202.120:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-08-12 01:40:05,080 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-83698808': 1, 'ctest-pod-03200971': 1} 2025-08-12 01:40:07,110 - DEBUG - [Pod ctest-pod-13192457] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.644 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.068 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.078 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.068/0.263/0.644 ms 2025-08-12 01:40:07,110 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-13192457 with expectation True passed 2025-08-12 01:40:09,159 - DEBUG - [Pod ctest-pod-01324802] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.430 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.090 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.080 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.080/0.200/0.430 ms 2025-08-12 01:40:09,159 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-01324802 with expectation True passed 2025-08-12 01:40:11,202 - DEBUG - [Pod ctest-pod-45836418] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.539 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.064 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.070 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.064/0.224/0.539 ms 2025-08-12 01:40:11,202 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-45836418 with expectation True passed 2025-08-12 01:40:11,252 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:40:11,252 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-08-12 01:40:11,252 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:40:11,252 - INFO - wget check of of http://10.47.255.251 from ctest-pod-13192457 passed 2025-08-12 01:40:11,285 - DEBUG - [Pod ctest-pod-13192457] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:40:11,285 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-08-12 01:40:11,285 - DEBUG - [Pod Pod ctest-pod-13192457] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:40:11,285 - INFO - wget check of of http://10.47.255.250 from ctest-pod-13192457 passed 2025-08-12 01:40:16,331 - DEBUG - [Pod ctest-pod-01324802] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-08-12 01:40:16,331 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-08-12 01:40:16,331 - DEBUG - [Pod Pod ctest-pod-01324802] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-08-12 01:40:16,331 - INFO - wget check of of http://10.47.255.250 from ctest-pod-01324802 passed 2025-08-12 01:40:16,375 - DEBUG - [Pod ctest-pod-91762201] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:40:16,375 - DEBUG - [Pod Pod ctest-pod-91762201] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-08-12 01:40:16,375 - DEBUG - [Pod Pod ctest-pod-91762201] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-08-12 01:40:16,375 - INFO - wget check of of http://10.47.255.251 from ctest-pod-91762201 passed 2025-08-12 01:40:16,375 - INFO - Deleting Network Policy : ingress-on-service 2025-08-12 01:40:16,376 - INFO - Deleting service : ctest-nginx-svc-76825044 2025-08-12 01:40:16,557 - DEBUG - No XMPP flaps were noticed during the test 2025-08-12 01:40:16,557 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:13] 2025-08-12 01:40:16,557 - INFO - -------------------------------------------------------------------------------- 2025-08-12 01:40:16,562 - INFO - Deleting pod ctest-new-default-98136278:ctest-pod-83698808 2025-08-12 01:40:16,562 - INFO - Deleting pod ctest-new-default-98136278:ctest-pod-03200971 2025-08-12 01:40:16,562 - INFO - Deleting pod ctest-non-default-47249989:ctest-pod-44131503 2025-08-12 01:40:16,563 - INFO - Deleting pod ctest-non-default-47249989:ctest-pod-19935220 2025-08-12 01:40:16,563 - INFO - Deleting pod ctest-new-default-98136278:ctest-pod-01324802 2025-08-12 01:40:16,563 - INFO - Deleting pod ctest-new-default-98136278:ctest-pod-13192457 2025-08-12 01:40:16,564 - INFO - Deleting pod ctest-non-default-47249989:ctest-pod-45836418 2025-08-12 01:40:16,564 - INFO - Deleting pod ctest-non-default-47249989:ctest-pod-91762201 2025-08-12 01:40:16,657 - WARNING - Pod uuid d2be86e9-a449-4f31-bc1b-09b3f7a220e6 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,658 - WARNING - Pod uuid 7bc7d613-c0d2-49ae-b9f4-574a98593467 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,658 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,659 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,660 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,661 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,661 - WARNING - Pod uuid 04bc01b2-cb63-4ea0-a98c-92245d66ad6c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:16,661 - WARNING - Pod uuid f373f0d8-050f-4f9b-877c-e49e7487fa7f is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:21,692 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:21,693 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:21,693 - DEBUG - Pod d2be86e9-a449-4f31-bc1b-09b3f7a220e6 is not in agent 10.0.0.249 VM list 2025-08-12 01:40:21,694 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:21,695 - DEBUG - Pod 7bc7d613-c0d2-49ae-b9f4-574a98593467 is not in agent 10.0.0.249 VM list 2025-08-12 01:40:21,698 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:21,698 - DEBUG - Pod f373f0d8-050f-4f9b-877c-e49e7487fa7f is not in agent 10.0.0.249 VM list 2025-08-12 01:40:21,700 - DEBUG - Pod 04bc01b2-cb63-4ea0-a98c-92245d66ad6c is not in agent 10.0.0.249 VM list 2025-08-12 01:40:21,713 - DEBUG - VMI ccdd262c-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:21,713 - INFO - Verified that pod ctest-pod-83698808 is removed in agent 2025-08-12 01:40:21,714 - DEBUG - VMI d065eeb4-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:21,714 - INFO - Verified that pod ctest-pod-19935220 is removed in agent 2025-08-12 01:40:21,714 - DEBUG - VMI cdf2b1b2-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:21,714 - INFO - Verified that pod ctest-pod-03200971 is removed in agent 2025-08-12 01:40:21,715 - DEBUG - VMI cff578be-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:21,715 - INFO - Verified that pod ctest-pod-44131503 is removed in agent 2025-08-12 01:40:26,709 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:26,711 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:26,711 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:26,711 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:31,727 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:31,728 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:31,729 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:31,729 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:36,748 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:36,748 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:36,749 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:36,750 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:41,765 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:41,767 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:41,767 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:41,768 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:46,838 - WARNING - Pod uuid eaba0f12-c404-4814-a60c-1b05de6e6e7c is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:46,839 - WARNING - Pod uuid 9a1207c0-0545-4e86-8186-4529668381ca is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:46,839 - WARNING - Pod uuid 653172c6-541e-494b-98af-d7e6b171d84b is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:46,840 - WARNING - Pod uuid be440c65-a1e7-420b-8b81-e8b029bec0b2 is still seen in agent 10.0.0.249 VM list 2025-08-12 01:40:51,854 - DEBUG - Pod be440c65-a1e7-420b-8b81-e8b029bec0b2 is not in agent 10.0.0.249 VM list 2025-08-12 01:40:51,857 - DEBUG - Pod 653172c6-541e-494b-98af-d7e6b171d84b is not in agent 10.0.0.249 VM list 2025-08-12 01:40:51,859 - DEBUG - Pod 9a1207c0-0545-4e86-8186-4529668381ca is not in agent 10.0.0.249 VM list 2025-08-12 01:40:51,860 - DEBUG - Pod eaba0f12-c404-4814-a60c-1b05de6e6e7c is not in agent 10.0.0.249 VM list 2025-08-12 01:40:51,872 - DEBUG - VMI d15669a2-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:51,872 - INFO - Verified that pod ctest-pod-13192457 is removed in agent 2025-08-12 01:40:51,874 - DEBUG - VMI d0e4db16-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:51,874 - INFO - Verified that pod ctest-pod-01324802 is removed in agent 2025-08-12 01:40:51,875 - DEBUG - VMI d247c6da-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:51,875 - INFO - Verified that pod ctest-pod-91762201 is removed in agent 2025-08-12 01:40:51,875 - DEBUG - VMI d1daf88e-771c-11f0-bea2-fa163e5be5c4 is removed from agent 10.0.0.249 2025-08-12 01:40:51,875 - INFO - Verified that pod ctest-pod-45836418 is removed in agent 2025-08-12 01:40:51,875 - INFO - Deleting namespace ctest-non-default-47249989 2025-08-12 01:40:51,896 - DEBUG - Namespace ctest-non-default-47249989 still in k8s 2025-08-12 01:40:53,899 - DEBUG - Namespace ctest-non-default-47249989 still in k8s 2025-08-12 01:40:55,904 - DEBUG - Namespace ctest-non-default-47249989 still in k8s 2025-08-12 01:40:57,908 - DEBUG - Namespace ctest-non-default-47249989 is not in k8s 2025-08-12 01:40:57,908 - DEBUG - No need to check project deletion in contrail-api 2025-08-12 01:40:57,908 - INFO - Deleting namespace ctest-new-default-98136278 2025-08-12 01:40:57,921 - DEBUG - Namespace ctest-new-default-98136278 still in k8s 2025-08-12 01:40:59,925 - DEBUG - Namespace ctest-new-default-98136278 still in k8s 2025-08-12 01:41:01,930 - DEBUG - Namespace ctest-new-default-98136278 still in k8s 2025-08-12 01:41:03,934 - DEBUG - Namespace ctest-new-default-98136278 is not in k8s 2025-08-12 01:41:03,934 - DEBUG - No need to check project deletion in contrail-api