2025-05-29 02:19:55,050 - INFO - Using existing project ['default-domain', 'k8s-default'](ddce9f14-e0f7-45ae-9780-d76233d03139) 2025-05-29 02:19:55,072 - DEBUG - Namespace ctest-new-default-08246453 not present 2025-05-29 02:19:55,089 - INFO - Created namespace ctest-new-default-08246453 2025-05-29 02:19:58,094 - DEBUG - Namespace ctest-non-default-76179017 not present 2025-05-29 02:19:58,111 - INFO - Created namespace ctest-non-default-76179017 2025-05-29 02:20:01,130 - DEBUG - Pod ctest-pod-54025312 not present 2025-05-29 02:20:01,131 - INFO - Creating Pod ctest-pod-54025312 2025-05-29 02:20:01,166 - DEBUG - Pod : ctest-pod-54025312 UUID is 7ee780aa-9de5-49e5-ae93-0f55d261a430 2025-05-29 02:20:01,170 - DEBUG - Pod ctest-pod-58276942 not present 2025-05-29 02:20:01,171 - INFO - Creating Pod ctest-pod-58276942 2025-05-29 02:20:01,208 - DEBUG - Pod : ctest-pod-58276942 UUID is f4d53492-4983-4059-b929-98e58c45c1cf 2025-05-29 02:20:01,210 - DEBUG - Pod ctest-pod-02353265 not present 2025-05-29 02:20:01,210 - INFO - Creating Pod ctest-pod-02353265 2025-05-29 02:20:01,246 - DEBUG - Pod : ctest-pod-02353265 UUID is 17b7fce9-13a3-469b-8ca2-e9a584ed0aaa 2025-05-29 02:20:01,254 - DEBUG - Pod ctest-pod-35787381 not present 2025-05-29 02:20:01,255 - INFO - Creating Pod ctest-pod-35787381 2025-05-29 02:20:01,289 - DEBUG - Pod : ctest-pod-35787381 UUID is 2f949077-537c-468d-9ef6-5bec8fc93a13 2025-05-29 02:20:01,291 - DEBUG - Pod ctest-pod-83814116 not present 2025-05-29 02:20:01,292 - INFO - Creating Pod ctest-pod-83814116 2025-05-29 02:20:01,312 - DEBUG - Pod : ctest-pod-83814116 UUID is 6a4c8492-4211-40cb-8074-573eeec879b1 2025-05-29 02:20:01,316 - DEBUG - Pod ctest-pod-63096155 not present 2025-05-29 02:20:01,316 - INFO - Creating Pod ctest-pod-63096155 2025-05-29 02:20:01,333 - DEBUG - Pod : ctest-pod-63096155 UUID is 63c2740d-5dd7-4e58-a9d8-d6e834301675 2025-05-29 02:20:01,335 - DEBUG - Pod ctest-pod-64118768 not present 2025-05-29 02:20:01,335 - INFO - Creating Pod ctest-pod-64118768 2025-05-29 02:20:01,352 - DEBUG - Pod : ctest-pod-64118768 UUID is c9a054a4-1e39-440f-9f6f-a39c44b62a14 2025-05-29 02:20:01,354 - DEBUG - Pod ctest-pod-61185996 not present 2025-05-29 02:20:01,354 - INFO - Creating Pod ctest-pod-61185996 2025-05-29 02:20:01,371 - DEBUG - Pod : ctest-pod-61185996 UUID is 10fb02f6-bdbf-479b-8c9e-f95ca57c11d1 2025-05-29 02:20:01,543 - INFO - Project default-domain:k8s-ctest-new-default-08246453 for Namespace ctest-new-default-08246453 is seen in contrail-api 2025-05-29 02:20:01,543 - DEBUG - Project uuid in Contrail is e378b9a6-e805-495a-8859-379dce905963 2025-05-29 02:20:01,725 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NamespaceDatabaseList?namespace_uuid=1c5b50bc-ed47-45c3-a0c4-466cc055c855 2025-05-29 02:20:01,730 - INFO - Namespace ctest-new-default-08246453 with uuid 1c5b50bc-ed47-45c3-a0c4-466cc055c855 found in kube manager 2025-05-29 02:20:01,730 - INFO - Namespace ctest-new-default-08246453 verification passed 2025-05-29 02:20:01,931 - INFO - Project default-domain:k8s-ctest-non-default-76179017 for Namespace ctest-non-default-76179017 is seen in contrail-api 2025-05-29 02:20:01,931 - DEBUG - Project uuid in Contrail is f1e5f0bb-2842-4681-8ced-d964a133cdba 2025-05-29 02:20:01,931 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NamespaceDatabaseList?namespace_uuid=35aa8f2f-b8b2-4069-9b85-8048362ead2b 2025-05-29 02:20:01,935 - INFO - Namespace ctest-non-default-76179017 with uuid 35aa8f2f-b8b2-4069-9b85-8048362ead2b found in kube manager 2025-05-29 02:20:01,935 - INFO - Namespace ctest-non-default-76179017 verification passed 2025-05-29 02:20:01,941 - DEBUG - Pod ctest-pod-54025312 not in running state.Currently in Pending 2025-05-29 02:20:06,955 - DEBUG - Pod ctest-pod-54025312 not in running state.Currently in Pending 2025-05-29 02:20:11,963 - INFO - Pod ctest-pod-54025312 is in running state.Got IP 10.47.255.251 2025-05-29 02:20:11,976 - DEBUG - Pod ctest-pod-54025312 has vmi 6d9fd23c-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:11,976 - INFO - Verified pod ctest-pod-54025312 in contrail-api 2025-05-29 02:20:11,989 - DEBUG - VMI 6d9fd23c-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:11,989 - INFO - Verified Pod ctest-pod-54025312 in agent 10.0.0.241 2025-05-29 02:20:11,990 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=7ee780aa-9de5-49e5-ae93-0f55d261a430 2025-05-29 02:20:11,994 - INFO - Pod ctest-pod-54025312 with uuid 7ee780aa-9de5-49e5-ae93-0f55d261a430 found in kube manager 2025-05-29 02:20:11,994 - INFO - Pod ctest-pod-54025312 verification passed 2025-05-29 02:20:11,999 - DEBUG - Pod ctest-pod-58276942 not in running state.Currently in Pending 2025-05-29 02:20:17,011 - INFO - Pod ctest-pod-58276942 is in running state.Got IP 10.47.255.250 2025-05-29 02:20:17,070 - DEBUG - Pod ctest-pod-58276942 has vmi 6e1c41c8-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:17,070 - INFO - Verified pod ctest-pod-58276942 in contrail-api 2025-05-29 02:20:17,084 - DEBUG - VMI 6e1c41c8-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:17,084 - INFO - Verified Pod ctest-pod-58276942 in agent 10.0.0.241 2025-05-29 02:20:17,084 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=f4d53492-4983-4059-b929-98e58c45c1cf 2025-05-29 02:20:17,091 - INFO - Pod ctest-pod-58276942 with uuid f4d53492-4983-4059-b929-98e58c45c1cf found in kube manager 2025-05-29 02:20:17,091 - INFO - Pod ctest-pod-58276942 verification passed 2025-05-29 02:20:17,101 - INFO - Pod ctest-pod-02353265 is in running state.Got IP 10.47.255.249 2025-05-29 02:20:17,113 - DEBUG - Pod ctest-pod-02353265 has vmi 6ebd0324-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:17,114 - INFO - Verified pod ctest-pod-02353265 in contrail-api 2025-05-29 02:20:17,127 - DEBUG - VMI 6ebd0324-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:17,127 - INFO - Verified Pod ctest-pod-02353265 in agent 10.0.0.241 2025-05-29 02:20:17,128 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=17b7fce9-13a3-469b-8ca2-e9a584ed0aaa 2025-05-29 02:20:17,131 - INFO - Pod ctest-pod-02353265 with uuid 17b7fce9-13a3-469b-8ca2-e9a584ed0aaa found in kube manager 2025-05-29 02:20:17,132 - INFO - Pod ctest-pod-02353265 verification passed 2025-05-29 02:20:17,136 - INFO - Pod ctest-pod-35787381 is in running state.Got IP 10.47.255.248 2025-05-29 02:20:17,191 - DEBUG - Pod ctest-pod-35787381 has vmi 6f38baaa-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:17,191 - INFO - Verified pod ctest-pod-35787381 in contrail-api 2025-05-29 02:20:17,204 - DEBUG - VMI 6f38baaa-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:17,205 - INFO - Verified Pod ctest-pod-35787381 in agent 10.0.0.241 2025-05-29 02:20:17,205 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=2f949077-537c-468d-9ef6-5bec8fc93a13 2025-05-29 02:20:17,210 - INFO - Pod ctest-pod-35787381 with uuid 2f949077-537c-468d-9ef6-5bec8fc93a13 found in kube manager 2025-05-29 02:20:17,210 - INFO - Pod ctest-pod-35787381 verification passed 2025-05-29 02:20:17,216 - INFO - Pod ctest-pod-83814116 is in running state.Got IP 10.47.255.247 2025-05-29 02:20:17,274 - DEBUG - Pod ctest-pod-83814116 has vmi 6fc761b0-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:17,274 - INFO - Verified pod ctest-pod-83814116 in contrail-api 2025-05-29 02:20:17,290 - DEBUG - VMI 6fc761b0-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:17,290 - INFO - Verified Pod ctest-pod-83814116 in agent 10.0.0.241 2025-05-29 02:20:17,290 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=6a4c8492-4211-40cb-8074-573eeec879b1 2025-05-29 02:20:17,295 - INFO - Pod ctest-pod-83814116 with uuid 6a4c8492-4211-40cb-8074-573eeec879b1 found in kube manager 2025-05-29 02:20:17,295 - INFO - Pod ctest-pod-83814116 verification passed 2025-05-29 02:20:17,379 - INFO - Pod ctest-pod-64118768 is in running state.Got IP 10.47.255.245 2025-05-29 02:20:17,394 - DEBUG - Pod ctest-pod-64118768 has vmi 70aaf86c-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:17,394 - INFO - Verified pod ctest-pod-64118768 in contrail-api 2025-05-29 02:20:17,407 - DEBUG - VMI 70aaf86c-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:17,407 - INFO - Verified Pod ctest-pod-64118768 in agent 10.0.0.241 2025-05-29 02:20:17,407 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=c9a054a4-1e39-440f-9f6f-a39c44b62a14 2025-05-29 02:20:17,412 - INFO - Pod ctest-pod-64118768 with uuid c9a054a4-1e39-440f-9f6f-a39c44b62a14 found in kube manager 2025-05-29 02:20:17,412 - INFO - Pod ctest-pod-64118768 verification passed 2025-05-29 02:20:17,417 - INFO - Pod ctest-pod-63096155 is in running state.Got IP 10.47.255.246 2025-05-29 02:20:17,474 - DEBUG - Pod ctest-pod-63096155 has vmi 7040890a-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:17,474 - INFO - Verified pod ctest-pod-63096155 in contrail-api 2025-05-29 02:20:17,488 - DEBUG - VMI 7040890a-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:17,488 - INFO - Verified Pod ctest-pod-63096155 in agent 10.0.0.241 2025-05-29 02:20:17,489 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=63c2740d-5dd7-4e58-a9d8-d6e834301675 2025-05-29 02:20:17,493 - INFO - Pod ctest-pod-63096155 with uuid 63c2740d-5dd7-4e58-a9d8-d6e834301675 found in kube manager 2025-05-29 02:20:17,493 - INFO - Pod ctest-pod-63096155 verification passed 2025-05-29 02:20:17,498 - DEBUG - Pod ctest-pod-61185996 not in running state.Currently in Pending 2025-05-29 02:20:22,504 - INFO - Pod ctest-pod-61185996 is in running state.Got IP 10.47.255.244 2025-05-29 02:20:22,519 - DEBUG - Pod ctest-pod-61185996 has vmi 71178c66-3c33-11f0-b248-fa163eac450b 2025-05-29 02:20:22,519 - INFO - Verified pod ctest-pod-61185996 in contrail-api 2025-05-29 02:20:22,535 - DEBUG - VMI 71178c66-3c33-11f0-b248-fa163eac450b is active in agent 10.0.0.241 2025-05-29 02:20:22,536 - INFO - Verified Pod ctest-pod-61185996 in agent 10.0.0.241 2025-05-29 02:20:22,536 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=10fb02f6-bdbf-479b-8c9e-f95ca57c11d1 2025-05-29 02:20:22,540 - INFO - Pod ctest-pod-61185996 with uuid 10fb02f6-bdbf-479b-8c9e-f95ca57c11d1 found in kube manager 2025-05-29 02:20:22,540 - INFO - Pod ctest-pod-61185996 verification passed 2025-05-29 02:20:22,541 - INFO - ================================================================================ 2025-05-29 02:20:22,541 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2025-05-29 02:20:22,541 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2025-05-29 02:20:22,755 - DEBUG - Nothing to compare xmpp stats {'10.0.0.241': {'10.0.0.22': '0'}} with 2025-05-29 02:20:22,755 - INFO - Initial checks done. Running the testcase now 2025-05-29 02:20:22,756 - INFO - 2025-05-29 02:20:22,810 - DEBUG - [Pod ctest-pod-54025312] Cmd: echo ctest-pod-54025312 > /usr/share/nginx/html/index.html, Output: 2025-05-29 02:20:22,877 - DEBUG - [Pod ctest-pod-58276942] Cmd: echo ctest-pod-58276942 > /usr/share/nginx/html/index.html, Output: 2025-05-29 02:20:22,932 - DEBUG - [Pod ctest-pod-02353265] Cmd: echo ctest-pod-02353265 > /usr/share/nginx/html/index.html, Output: 2025-05-29 02:20:22,988 - DEBUG - [Pod ctest-pod-35787381] Cmd: echo ctest-pod-35787381 > /usr/share/nginx/html/index.html, Output: 2025-05-29 02:20:22,991 - DEBUG - Service ctest-nginx-svc-86949045 not present 2025-05-29 02:20:22,992 - INFO - Creating service ctest-nginx-svc-86949045 2025-05-29 02:20:23,005 - INFO - Created Service ctest-nginx-svc-86949045 2025-05-29 02:20:23,012 - WARNING - Service UUID e8a6f825-5bda-44f1-bb97-3650387dccbe not yet found in contrail-api 2025-05-29 02:20:24,031 - INFO - Validated that Service ctest-nginx-svc-86949045 is seen in contrail-api 2025-05-29 02:20:24,032 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=e8a6f825-5bda-44f1-bb97-3650387dccbe 2025-05-29 02:20:24,036 - INFO - Service ctest-nginx-svc-86949045 with uuid e8a6f825-5bda-44f1-bb97-3650387dccbe found in kube manager 2025-05-29 02:20:24,036 - INFO - Service ctest-nginx-svc-86949045 verification passed 2025-05-29 02:20:24,098 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,098 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,098 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,157 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-54025312 2025-05-29 02:20:24,157 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,157 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-54025312 2025-05-29 02:20:24,238 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,238 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,238 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,295 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,295 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,295 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,359 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-58276942 2025-05-29 02:20:24,360 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,360 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-58276942 2025-05-29 02:20:24,360 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-54025312': 4, 'ctest-pod-58276942': 1} 2025-05-29 02:20:24,426 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,426 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,426 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,477 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-58276942 2025-05-29 02:20:24,477 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,477 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-58276942 2025-05-29 02:20:24,539 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,539 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:20:24,539 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:20:24,539 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-54025312': 1, 'ctest-pod-58276942': 2} 2025-05-29 02:20:24,542 - DEBUG - Network policy ingress-on-service not present 2025-05-29 02:20:24,542 - INFO - Creating Network Policy ingress-on-service 2025-05-29 02:20:24,551 - INFO - Network policy found in k8s 2025-05-29 02:20:24,552 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=fc7c6da9-446b-41eb-948f-4756f02041e1 2025-05-29 02:20:24,560 - INFO - Network Policy ingress-on-service with uuid fc7c6da9-446b-41eb-948f-4756f02041e1 found in kube manager 2025-05-29 02:20:24,599 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2025-05-29 02:20:24,600 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2025-05-29 02:20:24,600 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=fc7c6da9-446b-41eb-948f-4756f02041e1 2025-05-29 02:20:24,633 - WARNING - Network policy with name ingress-on-service not found in agent 2025-05-29 02:20:26,637 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=fc7c6da9-446b-41eb-948f-4756f02041e1 2025-05-29 02:20:26,670 - INFO - Network policy with name ingress-on-service found in agent 2025-05-29 02:20:26,670 - INFO - Network Policy ingress-on-service verification passed 2025-05-29 02:20:31,721 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:31,721 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:31,721 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:36,774 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:36,774 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:36,774 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:41,829 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:41,829 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:41,829 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:46,880 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:46,881 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:46,881 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:51,933 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:51,933 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:51,933 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:56,985 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:56,985 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:20:56,985 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:02,029 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:02,029 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:02,029 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:07,076 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:07,076 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:07,076 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:12,121 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:12,121 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:12,121 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:17,184 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:17,184 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:17,184 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:17,184 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-54025312': 0, 'ctest-pod-58276942': 0} 2025-05-29 02:21:17,244 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:21:17,244 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:21:17,244 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:21:17,300 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:21:17,300 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:21:17,300 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:21:17,300 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-54025312': 1, 'ctest-pod-58276942': 1} 2025-05-29 02:21:22,348 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:22,349 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:22,349 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:27,397 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:27,397 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:27,397 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:32,444 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:32,445 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:32,445 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:37,499 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:37,499 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:37,499 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:42,552 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:42,552 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:42,552 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:47,613 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:47,613 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:47,613 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:52,665 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:52,665 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:52,665 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:57,717 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:57,718 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:21:57,718 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:02,775 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:02,775 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:02,775 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:07,824 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:07,824 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:07,824 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:07,824 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-54025312': 0, 'ctest-pod-58276942': 0} 2025-05-29 02:22:09,876 - DEBUG - [Pod ctest-pod-63096155] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.058 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.082 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.085 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.082/0.408/1.058 ms 2025-05-29 02:22:09,876 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-63096155 with expectation True passed 2025-05-29 02:22:11,924 - DEBUG - [Pod ctest-pod-83814116] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.264 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.082 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.088 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.082/0.478/1.264 ms 2025-05-29 02:22:11,924 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-83814116 with expectation True passed 2025-05-29 02:22:13,981 - DEBUG - [Pod ctest-pod-64118768] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.101 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.084 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.114 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.084/0.433/1.101 ms 2025-05-29 02:22:13,982 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-64118768 with expectation True passed 2025-05-29 02:22:14,042 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:22:14,042 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-05-29 02:22:14,042 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:22:14,042 - INFO - wget check of of http://10.47.255.251 from ctest-pod-63096155 passed 2025-05-29 02:22:14,104 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:22:14,104 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-05-29 02:22:14,104 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:22:14,104 - INFO - wget check of of http://10.47.255.250 from ctest-pod-63096155 passed 2025-05-29 02:22:19,145 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-05-29 02:22:19,145 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-05-29 02:22:19,145 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-05-29 02:22:19,146 - INFO - wget check of of http://10.47.255.250 from ctest-pod-83814116 passed 2025-05-29 02:22:24,202 - DEBUG - [Pod ctest-pod-61185996] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-05-29 02:22:24,202 - DEBUG - [Pod Pod ctest-pod-61185996] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-05-29 02:22:24,202 - DEBUG - [Pod Pod ctest-pod-61185996] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-05-29 02:22:24,202 - INFO - wget check of of http://10.47.255.251 from ctest-pod-61185996 passed 2025-05-29 02:22:24,210 - INFO - Updating Network Policy ingress-on-service 2025-05-29 02:22:29,266 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:29,267 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:29,267 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:34,321 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:34,321 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:34,321 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:39,379 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:39,379 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:39,379 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:44,451 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:44,452 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:44,453 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:49,506 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:49,506 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:49,506 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:54,569 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:54,569 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:54,569 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:59,622 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:59,622 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:22:59,622 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:04,667 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:04,668 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:04,668 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:09,728 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:09,728 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:09,728 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:14,780 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:14,780 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:14,780 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) wget: download timed out 2025-05-29 02:23:14,780 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-54025312': 0, 'ctest-pod-58276942': 0} 2025-05-29 02:23:14,842 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:14,842 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:23:14,842 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:14,899 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:14,899 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:23:14,900 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:14,958 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:14,958 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:23:14,958 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: ctest-pod-58276942 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:15,005 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:15,005 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:23:15,005 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:15,005 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-54025312': 1, 'ctest-pod-58276942': 3} 2025-05-29 02:23:15,054 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:15,054 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:23:15,054 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: ctest-pod-54025312 Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-05-29 02:23:15,099 - DEBUG - [Pod ctest-pod-64118768] Cmd: wget http://10.111.198.238:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-58276942 2025-05-29 02:23:15,099 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd wget http://10.111.198.238:80/ -O - -T 5 -t 1 passed 2025-05-29 02:23:15,099 - DEBUG - [Pod Pod ctest-pod-64118768] Cmd output: Connecting to 10.111.198.238:80 (10.111.198.238:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-58276942 2025-05-29 02:23:15,099 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-54025312': 1, 'ctest-pod-58276942': 1} 2025-05-29 02:23:17,148 - DEBUG - [Pod ctest-pod-63096155] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.855 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.088 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.093 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.088/0.345/0.855 ms 2025-05-29 02:23:17,148 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-63096155 with expectation True passed 2025-05-29 02:23:19,196 - DEBUG - [Pod ctest-pod-83814116] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.135 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.077 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.092 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.077/0.434/1.135 ms 2025-05-29 02:23:19,196 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-83814116 with expectation True passed 2025-05-29 02:23:21,248 - DEBUG - [Pod ctest-pod-64118768] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.880 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.076 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.088 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.076/0.348/0.880 ms 2025-05-29 02:23:21,248 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-64118768 with expectation True passed 2025-05-29 02:23:21,298 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:23:21,298 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-05-29 02:23:21,298 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:23:21,299 - INFO - wget check of of http://10.47.255.251 from ctest-pod-63096155 passed 2025-05-29 02:23:21,355 - DEBUG - [Pod ctest-pod-63096155] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:23:21,355 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-05-29 02:23:21,355 - DEBUG - [Pod Pod ctest-pod-63096155] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:23:21,355 - INFO - wget check of of http://10.47.255.250 from ctest-pod-63096155 passed 2025-05-29 02:23:26,412 - DEBUG - [Pod ctest-pod-83814116] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-05-29 02:23:26,412 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-05-29 02:23:26,412 - DEBUG - [Pod Pod ctest-pod-83814116] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-05-29 02:23:26,412 - INFO - wget check of of http://10.47.255.250 from ctest-pod-83814116 passed 2025-05-29 02:23:26,466 - DEBUG - [Pod ctest-pod-61185996] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:23:26,467 - DEBUG - [Pod Pod ctest-pod-61185996] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-05-29 02:23:26,467 - DEBUG - [Pod Pod ctest-pod-61185996] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-05-29 02:23:26,467 - INFO - wget check of of http://10.47.255.251 from ctest-pod-61185996 passed 2025-05-29 02:23:26,467 - INFO - Deleting Network Policy : ingress-on-service 2025-05-29 02:23:26,467 - INFO - Deleting service : ctest-nginx-svc-86949045 2025-05-29 02:23:26,700 - DEBUG - No XMPP flaps were noticed during the test