2025-01-08 19:24:54,921 - INFO - Using existing project ['default-domain', 'k8s-default'](b98f5f30-bbbe-4544-b970-3e689771412c)
2025-01-08 19:24:54,943 - DEBUG - Namespace ctest-new-default-49801843 not present
2025-01-08 19:24:54,966 - INFO - Created namespace ctest-new-default-49801843
2025-01-08 19:24:57,971 - DEBUG - Namespace ctest-non-default-13386727 not present
2025-01-08 19:24:57,991 - INFO - Created namespace ctest-non-default-13386727
2025-01-08 19:25:01,010 - DEBUG - Pod ctest-pod-68989673 not present
2025-01-08 19:25:01,010 - INFO - Creating Pod ctest-pod-68989673
2025-01-08 19:25:01,043 - DEBUG - Pod : ctest-pod-68989673 UUID is 9aa42049-fa4c-42d1-bcbd-7d30e482e2fa
2025-01-08 19:25:01,048 - DEBUG - Pod ctest-pod-94221245 not present
2025-01-08 19:25:01,049 - INFO - Creating Pod ctest-pod-94221245
2025-01-08 19:25:01,067 - DEBUG - Pod : ctest-pod-94221245 UUID is c8030a5d-d868-42ea-8891-c7c76413be90
2025-01-08 19:25:01,076 - DEBUG - Pod ctest-pod-53948722 not present
2025-01-08 19:25:01,076 - INFO - Creating Pod ctest-pod-53948722
2025-01-08 19:25:01,111 - DEBUG - Pod : ctest-pod-53948722 UUID is 4b02f6b5-897d-46c3-8017-2f9becbb5399
2025-01-08 19:25:01,114 - DEBUG - Pod ctest-pod-16491957 not present
2025-01-08 19:25:01,114 - INFO - Creating Pod ctest-pod-16491957
2025-01-08 19:25:01,137 - DEBUG - Pod : ctest-pod-16491957 UUID is 9d7a0eff-7d4a-480d-87a7-6d4bd7f4f5f5
2025-01-08 19:25:01,140 - DEBUG - Pod ctest-pod-58584132 not present
2025-01-08 19:25:01,140 - INFO - Creating Pod ctest-pod-58584132
2025-01-08 19:25:01,161 - DEBUG - Pod : ctest-pod-58584132 UUID is 79012661-7821-4400-82cd-9079de682cc3
2025-01-08 19:25:01,167 - DEBUG - Pod ctest-pod-60323977 not present
2025-01-08 19:25:01,167 - INFO - Creating Pod ctest-pod-60323977
2025-01-08 19:25:01,192 - DEBUG - Pod : ctest-pod-60323977 UUID is 3f938f99-f6a7-49ec-afe8-de0a2afae8e6
2025-01-08 19:25:01,196 - DEBUG - Pod ctest-pod-15135621 not present
2025-01-08 19:25:01,196 - INFO - Creating Pod ctest-pod-15135621
2025-01-08 19:25:01,213 - DEBUG - Pod : ctest-pod-15135621 UUID is 87f5d259-3b79-4d97-b457-d8bc9e889302
2025-01-08 19:25:01,216 - DEBUG - Pod ctest-pod-59467778 not present
2025-01-08 19:25:01,216 - INFO - Creating Pod ctest-pod-59467778
2025-01-08 19:25:01,245 - DEBUG - Pod : ctest-pod-59467778 UUID is 625b107e-caf3-4a56-9881-42884865f270
2025-01-08 19:25:01,449 - INFO - Project default-domain:k8s-ctest-new-default-49801843 for Namespace ctest-new-default-49801843 is seen in contrail-api
2025-01-08 19:25:01,449 - DEBUG - Project uuid in Contrail is 4cab5979-563f-4093-a3ea-bd18b426470a
2025-01-08 19:25:01,610 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_NamespaceDatabaseList?namespace_uuid=22f111cf-99e2-4c61-bd6f-ed706b961e01
2025-01-08 19:25:01,632 - INFO - Namespace ctest-new-default-49801843 with uuid 22f111cf-99e2-4c61-bd6f-ed706b961e01 found in kube manager
2025-01-08 19:25:01,632 - INFO - Namespace ctest-new-default-49801843 verification passed
2025-01-08 19:25:01,881 - INFO - Project default-domain:k8s-ctest-non-default-13386727 for Namespace ctest-non-default-13386727 is seen in contrail-api
2025-01-08 19:25:01,881 - DEBUG - Project uuid in Contrail is 5c50f66f-a8e1-44a3-92cf-01c740f07abf
2025-01-08 19:25:01,881 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_NamespaceDatabaseList?namespace_uuid=a5f4a2b8-4a77-4845-9de2-c2fa0f8c6ec7
2025-01-08 19:25:01,892 - INFO - Namespace ctest-non-default-13386727 with uuid a5f4a2b8-4a77-4845-9de2-c2fa0f8c6ec7 found in kube manager
2025-01-08 19:25:01,894 - INFO - Namespace ctest-non-default-13386727 verification passed
2025-01-08 19:25:01,902 - DEBUG - Pod ctest-pod-68989673 not in running state.Currently in Pending
2025-01-08 19:25:06,911 - DEBUG - Pod ctest-pod-68989673 not in running state.Currently in Pending
2025-01-08 19:25:11,929 - INFO - Pod ctest-pod-68989673 is in running state.Got IP 10.47.255.251
2025-01-08 19:25:11,993 - DEBUG - Pod ctest-pod-68989673 has vmi 427ee35a-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:11,993 - INFO - Verified pod ctest-pod-68989673 in contrail-api
2025-01-08 19:25:12,009 - DEBUG - VMI 427ee35a-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:12,009 - INFO - Verified Pod ctest-pod-68989673 in agent 10.0.0.29
2025-01-08 19:25:12,009 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=9aa42049-fa4c-42d1-bcbd-7d30e482e2fa
2025-01-08 19:25:12,018 - INFO - Pod ctest-pod-68989673 with uuid 9aa42049-fa4c-42d1-bcbd-7d30e482e2fa found in kube manager
2025-01-08 19:25:12,018 - INFO - Pod ctest-pod-68989673 verification passed
2025-01-08 19:25:12,026 - DEBUG - Pod ctest-pod-94221245 not in running state.Currently in Pending
2025-01-08 19:25:17,035 - INFO - Pod ctest-pod-94221245 is in running state.Got IP 10.47.255.250
2025-01-08 19:25:17,056 - DEBUG - Pod ctest-pod-94221245 has vmi 4340a436-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:17,056 - INFO - Verified pod ctest-pod-94221245 in contrail-api
2025-01-08 19:25:17,076 - DEBUG - VMI 4340a436-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:17,077 - INFO - Verified Pod ctest-pod-94221245 in agent 10.0.0.29
2025-01-08 19:25:17,077 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=c8030a5d-d868-42ea-8891-c7c76413be90
2025-01-08 19:25:17,084 - INFO - Pod ctest-pod-94221245 with uuid c8030a5d-d868-42ea-8891-c7c76413be90 found in kube manager
2025-01-08 19:25:17,085 - INFO - Pod ctest-pod-94221245 verification passed
2025-01-08 19:25:17,093 - INFO - Pod ctest-pod-53948722 is in running state.Got IP 10.47.255.249
2025-01-08 19:25:17,114 - DEBUG - Pod ctest-pod-53948722 has vmi 43fa3c70-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:17,114 - INFO - Verified pod ctest-pod-53948722 in contrail-api
2025-01-08 19:25:17,131 - DEBUG - VMI 43fa3c70-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:17,131 - INFO - Verified Pod ctest-pod-53948722 in agent 10.0.0.29
2025-01-08 19:25:17,131 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=4b02f6b5-897d-46c3-8017-2f9becbb5399
2025-01-08 19:25:17,137 - INFO - Pod ctest-pod-53948722 with uuid 4b02f6b5-897d-46c3-8017-2f9becbb5399 found in kube manager
2025-01-08 19:25:17,137 - INFO - Pod ctest-pod-53948722 verification passed
2025-01-08 19:25:17,143 - INFO - Pod ctest-pod-16491957 is in running state.Got IP 10.47.255.248
2025-01-08 19:25:17,205 - DEBUG - Pod ctest-pod-16491957 has vmi 448ca1e6-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:17,205 - INFO - Verified pod ctest-pod-16491957 in contrail-api
2025-01-08 19:25:17,221 - DEBUG - VMI 448ca1e6-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:17,221 - INFO - Verified Pod ctest-pod-16491957 in agent 10.0.0.29
2025-01-08 19:25:17,222 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=9d7a0eff-7d4a-480d-87a7-6d4bd7f4f5f5
2025-01-08 19:25:17,227 - INFO - Pod ctest-pod-16491957 with uuid 9d7a0eff-7d4a-480d-87a7-6d4bd7f4f5f5 found in kube manager
2025-01-08 19:25:17,227 - INFO - Pod ctest-pod-16491957 verification passed
2025-01-08 19:25:17,232 - DEBUG - Pod ctest-pod-58584132 not in running state.Currently in Pending
2025-01-08 19:25:22,244 - INFO - Pod ctest-pod-58584132 is in running state.Got IP 10.47.255.247
2025-01-08 19:25:22,261 - DEBUG - Pod ctest-pod-58584132 has vmi 453670d6-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:22,261 - INFO - Verified pod ctest-pod-58584132 in contrail-api
2025-01-08 19:25:22,277 - DEBUG - VMI 453670d6-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:22,277 - INFO - Verified Pod ctest-pod-58584132 in agent 10.0.0.29
2025-01-08 19:25:22,277 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=79012661-7821-4400-82cd-9079de682cc3
2025-01-08 19:25:22,284 - INFO - Pod ctest-pod-58584132 with uuid 79012661-7821-4400-82cd-9079de682cc3 found in kube manager
2025-01-08 19:25:22,284 - INFO - Pod ctest-pod-58584132 verification passed
2025-01-08 19:25:22,292 - INFO - Pod ctest-pod-15135621 is in running state.Got IP 10.47.255.245
2025-01-08 19:25:22,358 - DEBUG - Pod ctest-pod-15135621 has vmi 4646555e-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:22,358 - INFO - Verified pod ctest-pod-15135621 in contrail-api
2025-01-08 19:25:22,373 - DEBUG - VMI 4646555e-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:22,373 - INFO - Verified Pod ctest-pod-15135621 in agent 10.0.0.29
2025-01-08 19:25:22,374 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=87f5d259-3b79-4d97-b457-d8bc9e889302
2025-01-08 19:25:22,380 - INFO - Pod ctest-pod-15135621 with uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 found in kube manager
2025-01-08 19:25:22,380 - INFO - Pod ctest-pod-15135621 verification passed
2025-01-08 19:25:22,397 - INFO - Pod ctest-pod-60323977 is in running state.Got IP 10.47.255.246
2025-01-08 19:25:22,424 - DEBUG - Pod ctest-pod-60323977 has vmi 45ba310a-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:22,424 - INFO - Verified pod ctest-pod-60323977 in contrail-api
2025-01-08 19:25:22,440 - DEBUG - VMI 45ba310a-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:22,441 - INFO - Verified Pod ctest-pod-60323977 in agent 10.0.0.29
2025-01-08 19:25:22,441 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=3f938f99-f6a7-49ec-afe8-de0a2afae8e6
2025-01-08 19:25:22,446 - INFO - Pod ctest-pod-60323977 with uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 found in kube manager
2025-01-08 19:25:22,446 - INFO - Pod ctest-pod-60323977 verification passed
2025-01-08 19:25:22,456 - INFO - Pod ctest-pod-59467778 is in running state.Got IP 10.47.255.244
2025-01-08 19:25:22,518 - DEBUG - Pod ctest-pod-59467778 has vmi 46d6cf26-cdf6-11ef-b818-fa163ee71078
2025-01-08 19:25:22,518 - INFO - Verified pod ctest-pod-59467778 in contrail-api
2025-01-08 19:25:22,536 - DEBUG - VMI 46d6cf26-cdf6-11ef-b818-fa163ee71078 is active in agent 10.0.0.29
2025-01-08 19:25:22,537 - INFO - Verified Pod ctest-pod-59467778 in agent 10.0.0.29
2025-01-08 19:25:22,537 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_VirtualMachineDatabaseList?x=625b107e-caf3-4a56-9881-42884865f270
2025-01-08 19:25:22,542 - INFO - Pod ctest-pod-59467778 with uuid 625b107e-caf3-4a56-9881-42884865f270 found in kube manager
2025-01-08 19:25:22,542 - INFO - Pod ctest-pod-59467778 verification passed
2025-01-08 19:25:22,543 - INFO - ================================================================================
2025-01-08 19:25:22,543 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service
2025-01-08 19:25:22,543 - INFO - TEST DESCRIPTION :
Verify that ingress rules on a pod work fine when pods are accessed via a service
Steps:
1. Create a service in namespace 1 over a pod which are endpoints of a service
2. Create a network policy to allow 1 pod of namespace 1 as ingress.
3. Verify that service respond correctly to the specified pod in the policy and
service do not respond to anyone else.
4. Extend the policy and add the namespaceselector as namespace 2.
5. Verify that service become accessible to the pods of namespace 2.
2025-01-08 19:25:22,806 - DEBUG - Nothing to compare xmpp stats {'10.0.0.29': {'10.0.0.31': '0'}} with
2025-01-08 19:25:22,807 - INFO - Initial checks done. Running the testcase now
2025-01-08 19:25:22,807 - INFO -
2025-01-08 19:25:22,872 - DEBUG - [Pod ctest-pod-68989673] Cmd: echo ctest-pod-68989673 > /usr/share/nginx/html/index.html, Output:
2025-01-08 19:25:22,943 - DEBUG - [Pod ctest-pod-94221245] Cmd: echo ctest-pod-94221245 > /usr/share/nginx/html/index.html, Output:
2025-01-08 19:25:22,999 - DEBUG - [Pod ctest-pod-53948722] Cmd: echo ctest-pod-53948722 > /usr/share/nginx/html/index.html, Output:
2025-01-08 19:25:23,073 - DEBUG - [Pod ctest-pod-16491957] Cmd: echo ctest-pod-16491957 > /usr/share/nginx/html/index.html, Output:
2025-01-08 19:25:23,078 - DEBUG - Service ctest-nginx-svc-76274022 not present
2025-01-08 19:25:23,079 - INFO - Creating service ctest-nginx-svc-76274022
2025-01-08 19:25:23,101 - INFO - Created Service ctest-nginx-svc-76274022
2025-01-08 19:25:23,124 - WARNING - Service UUID 5413152e-2119-4e15-8cfa-c3ebd83d5611 not yet found in contrail-api
2025-01-08 19:25:24,141 - INFO - Validated that Service ctest-nginx-svc-76274022 is seen in contrail-api
2025-01-08 19:25:24,141 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_LoadbalancerDatabaseList?x=5413152e-2119-4e15-8cfa-c3ebd83d5611
2025-01-08 19:25:24,148 - INFO - Service ctest-nginx-svc-76274022 with uuid 5413152e-2119-4e15-8cfa-c3ebd83d5611 found in kube manager
2025-01-08 19:25:24,148 - INFO - Service ctest-nginx-svc-76274022 verification passed
2025-01-08 19:25:25,248 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-94221245
2025-01-08 19:25:25,248 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:25:25,248 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-94221245
2025-01-08 19:25:25,306 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
ctest-pod-68989673
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:25:25,306 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:25:25,306 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
ctest-pod-68989673
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:25:25,306 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68989673': 1, 'ctest-pod-94221245': 1}
2025-01-08 19:25:25,356 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-94221245
2025-01-08 19:25:25,356 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:25:25,356 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-94221245
2025-01-08 19:25:25,410 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-68989673
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:25:25,411 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:25:25,411 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: ctest-pod-68989673
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:25:25,411 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68989673': 1, 'ctest-pod-94221245': 1}
2025-01-08 19:25:25,415 - DEBUG - Network policy ingress-on-service not present
2025-01-08 19:25:25,500 - INFO - Creating Network Policy ingress-on-service
2025-01-08 19:25:25,517 - INFO - Network policy found in k8s
2025-01-08 19:25:25,518 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a3bcf5fa-52a9-40d0-ab33-f9863f6e8f7d
2025-01-08 19:25:25,528 - INFO - Network Policy ingress-on-service with uuid a3bcf5fa-52a9-40d0-ab33-f9863f6e8f7d found in kube manager
2025-01-08 19:25:25,576 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent
2025-01-08 19:25:25,577 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8
2025-01-08 19:25:25,577 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a3bcf5fa-52a9-40d0-ab33-f9863f6e8f7d
2025-01-08 19:25:25,621 - WARNING - Network policy with name ingress-on-service not found in agent
2025-01-08 19:25:27,623 - DEBUG - Requesting: http://10.0.0.31:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a3bcf5fa-52a9-40d0-ab33-f9863f6e8f7d
2025-01-08 19:25:27,656 - INFO - Network policy with name ingress-on-service found in agent
2025-01-08 19:25:27,657 - INFO - Network Policy ingress-on-service verification passed
2025-01-08 19:25:32,712 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:32,712 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:32,712 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:37,781 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:37,781 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:37,781 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:42,848 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:42,848 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:42,848 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:47,913 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:47,914 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:47,914 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:52,969 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:52,969 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:52,969 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:58,027 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:58,027 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:25:58,027 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:03,087 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:03,087 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:03,087 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:08,146 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:08,146 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:08,147 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:13,225 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:13,225 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:13,225 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:14,897 - WARNING - ApiException is caught (0)
Reason: Connection is already closed.
. Retrying...
2025-01-08 19:26:24,957 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:24,958 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:24,958 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:24,958 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-68989673': 0, 'ctest-pod-94221245': 0}
2025-01-08 19:26:25,028 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:26:25,028 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:26:25,028 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:26:25,102 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:26:25,103 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:26:25,103 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:26:25,154 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:26:25,154 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:26:25,154 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:26:25,229 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-68989673
2025-01-08 19:26:25,230 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:26:25,230 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-68989673
2025-01-08 19:26:25,230 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68989673': 1, 'ctest-pod-94221245': 3}
2025-01-08 19:26:30,320 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:30,320 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:30,320 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:35,394 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:35,394 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:35,394 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:40,453 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:40,453 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:40,453 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:45,507 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:45,507 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:45,507 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:50,577 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:50,577 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:50,577 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:55,629 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:55,629 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:26:55,629 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:00,678 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:00,678 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:00,678 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:05,732 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:05,732 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:05,732 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:10,791 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:10,791 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:10,791 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:15,865 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:15,865 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:15,865 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:15,865 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-68989673': 0, 'ctest-pod-94221245': 0}
2025-01-08 19:27:17,961 - DEBUG - [Pod ctest-pod-60323977] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes
64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.255 ms
64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.062 ms
64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.083 ms
--- 10.47.255.247 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.062/0.466/1.255 ms
2025-01-08 19:27:17,961 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-60323977 with expectation True passed
2025-01-08 19:27:20,028 - DEBUG - [Pod ctest-pod-58584132] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.711 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.072 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.086 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.072/0.289/0.711 ms
2025-01-08 19:27:20,028 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-58584132 with expectation True passed
2025-01-08 19:27:22,083 - DEBUG - [Pod ctest-pod-15135621] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.634 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.069 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.110 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.069/0.604/1.634 ms
2025-01-08 19:27:22,083 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-15135621 with expectation True passed
2025-01-08 19:27:22,149 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:27:22,149 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed
2025-01-08 19:27:22,149 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:27:22,149 - INFO - wget check of of http://10.47.255.251 from ctest-pod-60323977 passed
2025-01-08 19:27:22,222 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:27:22,222 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed
2025-01-08 19:27:22,222 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:27:22,222 - INFO - wget check of of http://10.47.255.250 from ctest-pod-60323977 passed
2025-01-08 19:27:27,275 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-08 19:27:27,276 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-08 19:27:27,276 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-08 19:27:27,276 - INFO - wget check of of http://10.47.255.250 from ctest-pod-58584132 passed
2025-01-08 19:27:32,344 - DEBUG - [Pod ctest-pod-59467778] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
wget: download timed out
2025-01-08 19:27:32,344 - DEBUG - [Pod Pod ctest-pod-59467778] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80)
wget: download timed out
2025-01-08 19:27:32,344 - DEBUG - [Pod Pod ctest-pod-59467778] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
wget: download timed out
2025-01-08 19:27:32,344 - INFO - wget check of of http://10.47.255.251 from ctest-pod-59467778 passed
2025-01-08 19:27:32,356 - INFO - Updating Network Policy ingress-on-service
2025-01-08 19:27:37,425 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:37,425 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:37,425 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:42,484 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:42,484 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:42,484 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:47,546 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:47,546 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:47,546 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:52,613 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:52,615 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:52,615 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:57,685 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:57,686 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:27:57,686 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:02,746 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:02,746 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:02,746 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:07,809 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:07,809 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:07,809 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:12,868 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:12,868 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:12,868 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:17,928 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:17,928 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:17,928 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:22,998 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:22,999 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:22,999 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
wget: download timed out
2025-01-08 19:28:22,999 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-68989673': 0, 'ctest-pod-94221245': 0}
2025-01-08 19:28:23,061 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-68989673
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:28:23,061 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:28:23,061 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: ctest-pod-68989673
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:28:23,122 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:28:23,122 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:28:23,122 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: ctest-pod-94221245
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:28:23,122 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68989673': 1, 'ctest-pod-94221245': 1}
2025-01-08 19:28:23,178 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: ctest-pod-68989673
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:28:23,178 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:28:23,178 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: ctest-pod-68989673
Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-08 19:28:23,245 - DEBUG - [Pod ctest-pod-15135621] Cmd: wget http://10.99.117.109:80/ -O - -T 5 -t 1 , Output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-94221245
2025-01-08 19:28:23,245 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd wget http://10.99.117.109:80/ -O - -T 5 -t 1 passed
2025-01-08 19:28:23,245 - DEBUG - [Pod Pod ctest-pod-15135621] Cmd output: Connecting to 10.99.117.109:80 (10.99.117.109:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-94221245
2025-01-08 19:28:23,245 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68989673': 1, 'ctest-pod-94221245': 1}
2025-01-08 19:28:25,314 - DEBUG - [Pod ctest-pod-60323977] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes
64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.945 ms
64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.094 ms
64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.095 ms
--- 10.47.255.247 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.094/0.378/0.945 ms
2025-01-08 19:28:25,314 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-60323977 with expectation True passed
2025-01-08 19:28:27,377 - DEBUG - [Pod ctest-pod-58584132] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.282 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.065 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.087 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.065/0.478/1.282 ms
2025-01-08 19:28:27,377 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-58584132 with expectation True passed
2025-01-08 19:28:29,435 - DEBUG - [Pod ctest-pod-15135621] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.218 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.073 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.071 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.071/0.454/1.218 ms
2025-01-08 19:28:29,435 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-15135621 with expectation True passed
2025-01-08 19:28:29,494 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:28:29,494 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed
2025-01-08 19:28:29,494 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:28:29,494 - INFO - wget check of of http://10.47.255.251 from ctest-pod-60323977 passed
2025-01-08 19:28:29,558 - DEBUG - [Pod ctest-pod-60323977] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:28:29,558 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed
2025-01-08 19:28:29,558 - DEBUG - [Pod Pod ctest-pod-60323977] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:28:29,558 - INFO - wget check of of http://10.47.255.250 from ctest-pod-60323977 passed
2025-01-08 19:28:34,617 - DEBUG - [Pod ctest-pod-58584132] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-08 19:28:34,617 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-08 19:28:34,617 - DEBUG - [Pod Pod ctest-pod-58584132] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-08 19:28:34,618 - INFO - wget check of of http://10.47.255.250 from ctest-pod-58584132 passed
2025-01-08 19:28:34,690 - DEBUG - [Pod ctest-pod-59467778] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:28:34,690 - DEBUG - [Pod Pod ctest-pod-59467778] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed
2025-01-08 19:28:34,691 - DEBUG - [Pod Pod ctest-pod-59467778] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-08 19:28:34,691 - INFO - wget check of of http://10.47.255.251 from ctest-pod-59467778 passed
2025-01-08 19:28:34,691 - INFO - Deleting Network Policy : ingress-on-service
2025-01-08 19:28:34,692 - INFO - Deleting service : ctest-nginx-svc-76274022
2025-01-08 19:28:34,931 - DEBUG - No XMPP flaps were noticed during the test
2025-01-08 19:28:34,931 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:12]
2025-01-08 19:28:34,931 - INFO - --------------------------------------------------------------------------------
2025-01-08 19:28:34,936 - INFO - Deleting pod ctest-new-default-49801843:ctest-pod-68989673
2025-01-08 19:28:34,937 - INFO - Deleting pod ctest-new-default-49801843:ctest-pod-94221245
2025-01-08 19:28:34,937 - INFO - Deleting pod ctest-non-default-13386727:ctest-pod-53948722
2025-01-08 19:28:34,938 - INFO - Deleting pod ctest-non-default-13386727:ctest-pod-16491957
2025-01-08 19:28:34,938 - INFO - Deleting pod ctest-new-default-49801843:ctest-pod-58584132
2025-01-08 19:28:34,939 - INFO - Deleting pod ctest-new-default-49801843:ctest-pod-60323977
2025-01-08 19:28:34,939 - INFO - Deleting pod ctest-non-default-13386727:ctest-pod-15135621
2025-01-08 19:28:34,939 - INFO - Deleting pod ctest-non-default-13386727:ctest-pod-59467778
2025-01-08 19:28:35,012 - WARNING - Pod uuid 9aa42049-fa4c-42d1-bcbd-7d30e482e2fa is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,067 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,070 - WARNING - Pod uuid 9d7a0eff-7d4a-480d-87a7-6d4bd7f4f5f5 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,070 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,071 - WARNING - Pod uuid 4b02f6b5-897d-46c3-8017-2f9becbb5399 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,071 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,071 - WARNING - Pod uuid c8030a5d-d868-42ea-8891-c7c76413be90 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:35,072 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:40,020 - DEBUG - Pod 9aa42049-fa4c-42d1-bcbd-7d30e482e2fa is not in agent 10.0.0.29 VM list
2025-01-08 19:28:40,026 - DEBUG - VMI 427ee35a-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:28:40,026 - INFO - Verified that pod ctest-pod-68989673 is removed in agent
2025-01-08 19:28:40,098 - DEBUG - Pod 9d7a0eff-7d4a-480d-87a7-6d4bd7f4f5f5 is not in agent 10.0.0.29 VM list
2025-01-08 19:28:40,100 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:40,100 - DEBUG - Pod 4b02f6b5-897d-46c3-8017-2f9becbb5399 is not in agent 10.0.0.29 VM list
2025-01-08 19:28:40,104 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:40,104 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:40,105 - DEBUG - Pod c8030a5d-d868-42ea-8891-c7c76413be90 is not in agent 10.0.0.29 VM list
2025-01-08 19:28:40,106 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:40,127 - DEBUG - VMI 4340a436-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:28:40,127 - INFO - Verified that pod ctest-pod-94221245 is removed in agent
2025-01-08 19:28:40,127 - DEBUG - VMI 43fa3c70-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:28:40,127 - INFO - Verified that pod ctest-pod-53948722 is removed in agent
2025-01-08 19:28:40,128 - DEBUG - VMI 448ca1e6-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:28:40,128 - INFO - Verified that pod ctest-pod-16491957 is removed in agent
2025-01-08 19:28:45,110 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:45,120 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:45,121 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:45,122 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:50,119 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:50,135 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:50,137 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:50,137 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:55,127 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:55,150 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:55,152 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:28:55,152 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:00,140 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:00,165 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:00,166 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:00,276 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:05,166 - WARNING - Pod uuid 625b107e-caf3-4a56-9881-42884865f270 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:05,179 - WARNING - Pod uuid 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:05,180 - WARNING - Pod uuid 87f5d259-3b79-4d97-b457-d8bc9e889302 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:05,285 - WARNING - Pod uuid 79012661-7821-4400-82cd-9079de682cc3 is still seen in agent 10.0.0.29 VM list
2025-01-08 19:29:10,177 - DEBUG - Pod 625b107e-caf3-4a56-9881-42884865f270 is not in agent 10.0.0.29 VM list
2025-01-08 19:29:10,193 - DEBUG - VMI 46d6cf26-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:29:10,193 - INFO - Verified that pod ctest-pod-59467778 is removed in agent
2025-01-08 19:29:10,195 - DEBUG - Pod 3f938f99-f6a7-49ec-afe8-de0a2afae8e6 is not in agent 10.0.0.29 VM list
2025-01-08 19:29:10,198 - DEBUG - Pod 87f5d259-3b79-4d97-b457-d8bc9e889302 is not in agent 10.0.0.29 VM list
2025-01-08 19:29:10,211 - DEBUG - VMI 4646555e-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:29:10,211 - INFO - Verified that pod ctest-pod-15135621 is removed in agent
2025-01-08 19:29:10,212 - DEBUG - VMI 45ba310a-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:29:10,212 - INFO - Verified that pod ctest-pod-60323977 is removed in agent
2025-01-08 19:29:10,294 - DEBUG - Pod 79012661-7821-4400-82cd-9079de682cc3 is not in agent 10.0.0.29 VM list
2025-01-08 19:29:10,303 - DEBUG - VMI 453670d6-cdf6-11ef-b818-fa163ee71078 is removed from agent 10.0.0.29
2025-01-08 19:29:10,303 - INFO - Verified that pod ctest-pod-58584132 is removed in agent
2025-01-08 19:29:10,303 - INFO - Deleting namespace ctest-non-default-13386727
2025-01-08 19:29:10,326 - DEBUG - Namespace ctest-non-default-13386727 still in k8s
2025-01-08 19:29:12,331 - DEBUG - Namespace ctest-non-default-13386727 still in k8s
2025-01-08 19:29:14,337 - DEBUG - Namespace ctest-non-default-13386727 still in k8s
2025-01-08 19:29:16,341 - DEBUG - Namespace ctest-non-default-13386727 is not in k8s
2025-01-08 19:29:16,341 - DEBUG - No need to check project deletion in contrail-api
2025-01-08 19:29:16,341 - INFO - Deleting namespace ctest-new-default-49801843
2025-01-08 19:29:16,353 - DEBUG - Namespace ctest-new-default-49801843 still in k8s
2025-01-08 19:29:18,358 - DEBUG - Namespace ctest-new-default-49801843 still in k8s
2025-01-08 19:29:20,363 - DEBUG - Namespace ctest-new-default-49801843 still in k8s
2025-01-08 19:29:22,367 - DEBUG - Namespace ctest-new-default-49801843 is not in k8s
2025-01-08 19:29:22,367 - DEBUG - No need to check project deletion in contrail-api