2025-07-23 15:35:03,015 - INFO - Using existing project ['default-domain', 'k8s-default'](e2a8c8f1-f6fb-41e3-8443-56cebd724e3d) 2025-07-23 15:35:03,042 - DEBUG - Namespace ctest-new-default-86078139 not present 2025-07-23 15:35:03,056 - INFO - Created namespace ctest-new-default-86078139 2025-07-23 15:35:06,061 - DEBUG - Namespace ctest-non-default-23262455 not present 2025-07-23 15:35:06,068 - INFO - Created namespace ctest-non-default-23262455 2025-07-23 15:35:09,090 - DEBUG - Pod ctest-pod-68810910 not present 2025-07-23 15:35:09,090 - INFO - Creating Pod ctest-pod-68810910 2025-07-23 15:35:09,109 - DEBUG - Pod : ctest-pod-68810910 UUID is 8765e13f-ce62-4e50-8918-a7e5be9f42e5 2025-07-23 15:35:09,115 - DEBUG - Pod ctest-pod-14502408 not present 2025-07-23 15:35:09,116 - INFO - Creating Pod ctest-pod-14502408 2025-07-23 15:35:09,142 - DEBUG - Pod : ctest-pod-14502408 UUID is a1b86db8-f295-4a05-a0c3-5bc74e209daf 2025-07-23 15:35:09,146 - DEBUG - Pod ctest-pod-12527673 not present 2025-07-23 15:35:09,147 - INFO - Creating Pod ctest-pod-12527673 2025-07-23 15:35:09,169 - DEBUG - Pod : ctest-pod-12527673 UUID is 97aab466-138a-4f1f-bc12-453376e66111 2025-07-23 15:35:09,173 - DEBUG - Pod ctest-pod-84632467 not present 2025-07-23 15:35:09,173 - INFO - Creating Pod ctest-pod-84632467 2025-07-23 15:35:09,191 - DEBUG - Pod : ctest-pod-84632467 UUID is 0b3d65e6-68c0-49a5-aefa-23a19f7fa41a 2025-07-23 15:35:09,195 - DEBUG - Pod ctest-pod-00885831 not present 2025-07-23 15:35:09,195 - INFO - Creating Pod ctest-pod-00885831 2025-07-23 15:35:09,212 - DEBUG - Pod : ctest-pod-00885831 UUID is 327e4e13-1a92-4b19-a616-7cbde969eede 2025-07-23 15:35:09,216 - DEBUG - Pod ctest-pod-48554228 not present 2025-07-23 15:35:09,216 - INFO - Creating Pod ctest-pod-48554228 2025-07-23 15:35:09,234 - DEBUG - Pod : ctest-pod-48554228 UUID is 6697c5f2-8ae7-4e82-81ec-beacf1b0521d 2025-07-23 15:35:09,237 - DEBUG - Pod ctest-pod-60596179 not present 2025-07-23 15:35:09,238 - INFO - Creating Pod ctest-pod-60596179 2025-07-23 15:35:09,255 - DEBUG - Pod : ctest-pod-60596179 UUID is c966a941-a84a-480e-9b7e-5bc651f10a8f 2025-07-23 15:35:09,257 - DEBUG - Pod ctest-pod-48597499 not present 2025-07-23 15:35:09,258 - INFO - Creating Pod ctest-pod-48597499 2025-07-23 15:35:09,278 - DEBUG - Pod : ctest-pod-48597499 UUID is b5664ebd-f838-4ebc-8cc2-b36661e967d2 2025-07-23 15:35:09,463 - INFO - Project default-domain:k8s-ctest-new-default-86078139 for Namespace ctest-new-default-86078139 is seen in contrail-api 2025-07-23 15:35:09,463 - DEBUG - Project uuid in Contrail is a24f2f71-a44d-4365-83c0-1764f3ad1fdf 2025-07-23 15:35:09,612 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NamespaceDatabaseList?namespace_uuid=e54c4e30-53d5-4d7c-a75e-744e3a2f8298 2025-07-23 15:35:09,616 - INFO - Namespace ctest-new-default-86078139 with uuid e54c4e30-53d5-4d7c-a75e-744e3a2f8298 found in kube manager 2025-07-23 15:35:09,617 - INFO - Namespace ctest-new-default-86078139 verification passed 2025-07-23 15:35:09,863 - INFO - Project default-domain:k8s-ctest-non-default-23262455 for Namespace ctest-non-default-23262455 is seen in contrail-api 2025-07-23 15:35:09,863 - DEBUG - Project uuid in Contrail is 068dacf0-de92-4317-b004-ef99017ae655 2025-07-23 15:35:09,863 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NamespaceDatabaseList?namespace_uuid=072083f6-3082-464b-9f3a-dbf42589569a 2025-07-23 15:35:09,869 - INFO - Namespace ctest-non-default-23262455 with uuid 072083f6-3082-464b-9f3a-dbf42589569a found in kube manager 2025-07-23 15:35:09,869 - INFO - Namespace ctest-non-default-23262455 verification passed 2025-07-23 15:35:09,875 - DEBUG - Pod ctest-pod-68810910 not in running state.Currently in Pending 2025-07-23 15:35:14,887 - DEBUG - Pod ctest-pod-68810910 not in running state.Currently in Pending 2025-07-23 15:35:19,898 - DEBUG - Pod ctest-pod-68810910 not in running state.Currently in Pending 2025-07-23 15:35:24,905 - DEBUG - Pod ctest-pod-68810910 not in running state.Currently in Pending 2025-07-23 15:35:29,912 - INFO - Pod ctest-pod-68810910 is in running state.Got IP 10.47.255.248 2025-07-23 15:35:29,925 - DEBUG - Pod ctest-pod-68810910 has vmi a201b1ee-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:35:29,926 - INFO - Verified pod ctest-pod-68810910 in contrail-api 2025-07-23 15:35:29,936 - DEBUG - VMI a201b1ee-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:35:29,936 - INFO - Verified Pod ctest-pod-68810910 in agent 10.0.0.254 2025-07-23 15:35:29,936 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=8765e13f-ce62-4e50-8918-a7e5be9f42e5 2025-07-23 15:35:29,941 - INFO - Pod ctest-pod-68810910 with uuid 8765e13f-ce62-4e50-8918-a7e5be9f42e5 found in kube manager 2025-07-23 15:35:29,941 - INFO - Pod ctest-pod-68810910 verification passed 2025-07-23 15:35:29,946 - DEBUG - Pod ctest-pod-14502408 not in running state.Currently in Pending 2025-07-23 15:35:34,954 - INFO - Pod ctest-pod-14502408 is in running state.Got IP 10.47.255.247 2025-07-23 15:35:34,971 - DEBUG - Pod ctest-pod-14502408 has vmi a38fecba-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:35:34,971 - INFO - Verified pod ctest-pod-14502408 in contrail-api 2025-07-23 15:35:34,981 - DEBUG - VMI a38fecba-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:35:34,982 - INFO - Verified Pod ctest-pod-14502408 in agent 10.0.0.254 2025-07-23 15:35:34,982 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=a1b86db8-f295-4a05-a0c3-5bc74e209daf 2025-07-23 15:35:34,988 - INFO - Pod ctest-pod-14502408 with uuid a1b86db8-f295-4a05-a0c3-5bc74e209daf found in kube manager 2025-07-23 15:35:34,988 - INFO - Pod ctest-pod-14502408 verification passed 2025-07-23 15:35:34,994 - DEBUG - Pod ctest-pod-12527673 not in running state.Currently in Pending 2025-07-23 15:35:40,001 - INFO - Pod ctest-pod-12527673 is in running state.Got IP 10.47.255.246 2025-07-23 15:35:40,017 - DEBUG - Pod ctest-pod-12527673 has vmi a6552bcc-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:35:40,017 - INFO - Verified pod ctest-pod-12527673 in contrail-api 2025-07-23 15:35:40,030 - DEBUG - VMI a6552bcc-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:35:40,030 - INFO - Verified Pod ctest-pod-12527673 in agent 10.0.0.254 2025-07-23 15:35:40,030 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=97aab466-138a-4f1f-bc12-453376e66111 2025-07-23 15:35:40,035 - INFO - Pod ctest-pod-12527673 with uuid 97aab466-138a-4f1f-bc12-453376e66111 found in kube manager 2025-07-23 15:35:40,035 - INFO - Pod ctest-pod-12527673 verification passed 2025-07-23 15:35:40,041 - DEBUG - Pod ctest-pod-84632467 not in running state.Currently in Pending 2025-07-23 15:35:45,047 - INFO - Pod ctest-pod-84632467 is in running state.Got IP 10.47.255.245 2025-07-23 15:35:45,062 - DEBUG - Pod ctest-pod-84632467 has vmi a8772d1a-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:35:45,062 - INFO - Verified pod ctest-pod-84632467 in contrail-api 2025-07-23 15:35:45,072 - DEBUG - VMI a8772d1a-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:35:45,072 - INFO - Verified Pod ctest-pod-84632467 in agent 10.0.0.254 2025-07-23 15:35:45,073 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=0b3d65e6-68c0-49a5-aefa-23a19f7fa41a 2025-07-23 15:35:45,077 - INFO - Pod ctest-pod-84632467 with uuid 0b3d65e6-68c0-49a5-aefa-23a19f7fa41a found in kube manager 2025-07-23 15:35:45,077 - INFO - Pod ctest-pod-84632467 verification passed 2025-07-23 15:35:45,082 - DEBUG - Pod ctest-pod-00885831 not in running state.Currently in Pending 2025-07-23 15:35:50,091 - DEBUG - Pod ctest-pod-00885831 not in running state.Currently in Pending 2025-07-23 15:35:55,098 - DEBUG - Pod ctest-pod-00885831 not in running state.Currently in Pending 2025-07-23 15:36:00,106 - INFO - Pod ctest-pod-00885831 is in running state.Got IP 10.47.255.244 2025-07-23 15:36:00,118 - DEBUG - Pod ctest-pod-00885831 has vmi aaa39880-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:36:00,118 - INFO - Verified pod ctest-pod-00885831 in contrail-api 2025-07-23 15:36:00,130 - DEBUG - VMI aaa39880-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:36:00,131 - INFO - Verified Pod ctest-pod-00885831 in agent 10.0.0.254 2025-07-23 15:36:00,131 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=327e4e13-1a92-4b19-a616-7cbde969eede 2025-07-23 15:36:00,135 - INFO - Pod ctest-pod-00885831 with uuid 327e4e13-1a92-4b19-a616-7cbde969eede found in kube manager 2025-07-23 15:36:00,135 - INFO - Pod ctest-pod-00885831 verification passed 2025-07-23 15:36:00,141 - INFO - Pod ctest-pod-60596179 is in running state.Got IP 10.47.255.242 2025-07-23 15:36:00,156 - DEBUG - Pod ctest-pod-60596179 has vmi aed33b68-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:36:00,157 - INFO - Verified pod ctest-pod-60596179 in contrail-api 2025-07-23 15:36:00,168 - DEBUG - VMI aed33b68-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:36:00,169 - INFO - Verified Pod ctest-pod-60596179 in agent 10.0.0.254 2025-07-23 15:36:00,169 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=c966a941-a84a-480e-9b7e-5bc651f10a8f 2025-07-23 15:36:00,173 - INFO - Pod ctest-pod-60596179 with uuid c966a941-a84a-480e-9b7e-5bc651f10a8f found in kube manager 2025-07-23 15:36:00,173 - INFO - Pod ctest-pod-60596179 verification passed 2025-07-23 15:36:00,178 - INFO - Pod ctest-pod-48554228 is in running state.Got IP 10.47.255.243 2025-07-23 15:36:00,190 - DEBUG - Pod ctest-pod-48554228 has vmi ade95e6c-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:36:00,190 - INFO - Verified pod ctest-pod-48554228 in contrail-api 2025-07-23 15:36:00,202 - DEBUG - VMI ade95e6c-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:36:00,202 - INFO - Verified Pod ctest-pod-48554228 in agent 10.0.0.254 2025-07-23 15:36:00,202 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=6697c5f2-8ae7-4e82-81ec-beacf1b0521d 2025-07-23 15:36:00,206 - INFO - Pod ctest-pod-48554228 with uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d found in kube manager 2025-07-23 15:36:00,206 - INFO - Pod ctest-pod-48554228 verification passed 2025-07-23 15:36:00,212 - INFO - Pod ctest-pod-48597499 is in running state.Got IP 10.47.255.241 2025-07-23 15:36:00,224 - DEBUG - Pod ctest-pod-48597499 has vmi b220e5a4-67da-11f0-be54-fa163e7cd4f5 2025-07-23 15:36:00,224 - INFO - Verified pod ctest-pod-48597499 in contrail-api 2025-07-23 15:36:00,237 - DEBUG - VMI b220e5a4-67da-11f0-be54-fa163e7cd4f5 is active in agent 10.0.0.254 2025-07-23 15:36:00,237 - INFO - Verified Pod ctest-pod-48597499 in agent 10.0.0.254 2025-07-23 15:36:00,237 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_VirtualMachineDatabaseList?x=b5664ebd-f838-4ebc-8cc2-b36661e967d2 2025-07-23 15:36:00,241 - INFO - Pod ctest-pod-48597499 with uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 found in kube manager 2025-07-23 15:36:00,242 - INFO - Pod ctest-pod-48597499 verification passed 2025-07-23 15:36:00,242 - INFO - ================================================================================ 2025-07-23 15:36:00,242 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2025-07-23 15:36:00,242 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2025-07-23 15:36:00,759 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.18': '0'}} with 2025-07-23 15:36:00,759 - INFO - Initial checks done. Running the testcase now 2025-07-23 15:36:00,760 - INFO - 2025-07-23 15:36:00,826 - DEBUG - [Pod ctest-pod-68810910] Cmd: echo ctest-pod-68810910 > /usr/share/nginx/html/index.html, Output: 2025-07-23 15:36:00,879 - DEBUG - [Pod ctest-pod-14502408] Cmd: echo ctest-pod-14502408 > /usr/share/nginx/html/index.html, Output: 2025-07-23 15:36:00,922 - DEBUG - [Pod ctest-pod-12527673] Cmd: echo ctest-pod-12527673 > /usr/share/nginx/html/index.html, Output: 2025-07-23 15:36:00,962 - DEBUG - [Pod ctest-pod-84632467] Cmd: echo ctest-pod-84632467 > /usr/share/nginx/html/index.html, Output: 2025-07-23 15:36:00,966 - DEBUG - Service ctest-nginx-svc-59434346 not present 2025-07-23 15:36:00,966 - INFO - Creating service ctest-nginx-svc-59434346 2025-07-23 15:36:00,979 - INFO - Created Service ctest-nginx-svc-59434346 2025-07-23 15:36:00,992 - WARNING - Service UUID 49d4e717-7965-4f55-896d-4907de2d76ac not yet found in contrail-api 2025-07-23 15:36:02,000 - WARNING - Service UUID 49d4e717-7965-4f55-896d-4907de2d76ac not yet found in contrail-api 2025-07-23 15:36:03,005 - WARNING - Service UUID 49d4e717-7965-4f55-896d-4907de2d76ac not yet found in contrail-api 2025-07-23 15:36:04,011 - WARNING - Service UUID 49d4e717-7965-4f55-896d-4907de2d76ac not yet found in contrail-api 2025-07-23 15:36:05,018 - INFO - Validated that Service ctest-nginx-svc-59434346 is seen in contrail-api 2025-07-23 15:36:05,019 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_LoadbalancerDatabaseList?x=49d4e717-7965-4f55-896d-4907de2d76ac 2025-07-23 15:36:05,023 - INFO - Service ctest-nginx-svc-59434346 with uuid 49d4e717-7965-4f55-896d-4907de2d76ac found in kube manager 2025-07-23 15:36:05,023 - INFO - Service ctest-nginx-svc-59434346 verification passed 2025-07-23 15:36:06,118 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:36:06,118 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:36:06,118 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:36:06,159 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:36:06,159 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:36:06,159 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:36:06,159 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68810910': 1, 'ctest-pod-14502408': 1} 2025-07-23 15:36:06,205 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:36:06,205 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:36:06,205 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:36:06,244 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:36:06,245 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:36:06,245 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:36:06,281 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:36:06,281 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:36:06,281 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:36:06,282 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68810910': 1, 'ctest-pod-14502408': 2} 2025-07-23 15:36:06,285 - DEBUG - Network policy ingress-on-service not present 2025-07-23 15:36:06,287 - INFO - Creating Network Policy ingress-on-service 2025-07-23 15:36:06,296 - INFO - Network policy found in k8s 2025-07-23 15:36:06,296 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a0c5ca36-139a-469d-8961-642fa2a54798 2025-07-23 15:36:06,302 - INFO - Network Policy ingress-on-service with uuid a0c5ca36-139a-469d-8961-642fa2a54798 found in kube manager 2025-07-23 15:36:06,395 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2025-07-23 15:36:06,395 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2025-07-23 15:36:06,395 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a0c5ca36-139a-469d-8961-642fa2a54798 2025-07-23 15:36:06,422 - WARNING - Network policy with name ingress-on-service not found in agent 2025-07-23 15:36:08,422 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a0c5ca36-139a-469d-8961-642fa2a54798 2025-07-23 15:36:08,451 - WARNING - Network policy with name ingress-on-service not found in agent 2025-07-23 15:36:10,452 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a0c5ca36-139a-469d-8961-642fa2a54798 2025-07-23 15:36:10,481 - WARNING - Network policy with name ingress-on-service not found in agent 2025-07-23 15:36:12,482 - DEBUG - Requesting: http://10.0.0.254:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=a0c5ca36-139a-469d-8961-642fa2a54798 2025-07-23 15:36:12,511 - INFO - Network policy with name ingress-on-service found in agent 2025-07-23 15:36:12,512 - INFO - Network Policy ingress-on-service verification passed 2025-07-23 15:36:17,553 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:17,553 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:17,553 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:22,593 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:22,593 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:22,593 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:27,634 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:27,635 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:27,635 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:32,684 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:32,684 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:32,684 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:37,725 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:37,725 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:37,725 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:42,787 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:42,787 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:42,787 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:47,836 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:47,836 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:47,836 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:52,876 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:52,876 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:52,876 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:57,916 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:57,916 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:36:57,916 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:02,966 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:02,966 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:02,966 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:02,966 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-68810910': 0, 'ctest-pod-14502408': 0} 2025-07-23 15:37:03,015 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-68810910 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:37:03,015 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:37:03,015 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: ctest-pod-68810910 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:37:03,056 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:37:03,056 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:37:03,056 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:37:03,099 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:37:03,099 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:37:03,099 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:37:03,138 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:37:03,139 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:37:03,139 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:37:03,139 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68810910': 3, 'ctest-pod-14502408': 1} 2025-07-23 15:37:08,186 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:08,186 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:08,186 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:13,226 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:13,226 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:13,226 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:18,267 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:18,267 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:18,267 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:23,312 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:23,312 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:23,312 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:28,364 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:28,364 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:28,364 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:33,413 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:33,413 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:33,414 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:38,452 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:38,453 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:38,453 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:43,506 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:43,506 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:43,506 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:48,552 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:48,552 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:48,552 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:53,593 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:53,593 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:53,593 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:37:53,593 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-68810910': 0, 'ctest-pod-14502408': 0} 2025-07-23 15:37:55,635 - DEBUG - [Pod ctest-pod-48554228] Cmd: ping -c 3 10.47.255.244, Output: PING 10.47.255.244 (10.47.255.244): 56 data bytes 64 bytes from 10.47.255.244: seq=0 ttl=63 time=0.742 ms 64 bytes from 10.47.255.244: seq=1 ttl=63 time=0.081 ms 64 bytes from 10.47.255.244: seq=2 ttl=63 time=0.084 ms --- 10.47.255.244 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.081/0.302/0.742 ms 2025-07-23 15:37:55,635 - INFO - Ping check to IP 10.47.255.244 from pod ctest-pod-48554228 with expectation True passed 2025-07-23 15:37:57,677 - DEBUG - [Pod ctest-pod-00885831] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=1.052 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.069 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.071 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.069/0.397/1.052 ms 2025-07-23 15:37:57,677 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-00885831 with expectation True passed 2025-07-23 15:37:59,719 - DEBUG - [Pod ctest-pod-60596179] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=0.402 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.074 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.093 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.074/0.189/0.402 ms 2025-07-23 15:37:59,719 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-60596179 with expectation True passed 2025-07-23 15:37:59,770 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:37:59,770 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 passed 2025-07-23 15:37:59,770 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:37:59,770 - INFO - wget check of of http://10.47.255.248 from ctest-pod-48554228 passed 2025-07-23 15:37:59,810 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:37:59,810 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 passed 2025-07-23 15:37:59,810 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:37:59,810 - INFO - wget check of of http://10.47.255.247 from ctest-pod-48554228 passed 2025-07-23 15:38:04,852 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2025-07-23 15:38:04,853 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2025-07-23 15:38:04,853 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2025-07-23 15:38:04,853 - INFO - wget check of of http://10.47.255.247 from ctest-pod-00885831 passed 2025-07-23 15:38:09,897 - DEBUG - [Pod ctest-pod-48597499] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) wget: download timed out 2025-07-23 15:38:09,897 - DEBUG - [Pod Pod ctest-pod-48597499] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.248 (10.47.255.248:80) wget: download timed out 2025-07-23 15:38:09,897 - DEBUG - [Pod Pod ctest-pod-48597499] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) wget: download timed out 2025-07-23 15:38:09,897 - INFO - wget check of of http://10.47.255.248 from ctest-pod-48597499 passed 2025-07-23 15:38:09,908 - INFO - Updating Network Policy ingress-on-service 2025-07-23 15:38:14,963 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:14,964 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:14,964 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:20,005 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:20,005 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:20,005 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:25,047 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:25,047 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:25,047 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:30,088 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:30,088 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:30,088 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:35,128 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:35,128 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:35,128 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:40,171 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:40,171 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:40,171 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:45,217 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:45,217 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:45,217 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:50,257 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:50,257 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:50,257 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:55,298 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:55,298 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:38:55,298 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:39:00,345 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:39:00,345 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:39:00,345 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) wget: download timed out 2025-07-23 15:39:00,345 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-68810910': 0, 'ctest-pod-14502408': 0} 2025-07-23 15:39:00,398 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,398 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,398 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,439 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:39:00,440 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,440 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:39:00,484 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,484 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,484 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,524 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:39:00,525 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,525 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14502408 2025-07-23 15:39:00,563 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,563 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,563 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,563 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68810910': 1, 'ctest-pod-14502408': 4} 2025-07-23 15:39:00,604 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-68810910 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,604 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,604 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: ctest-pod-68810910 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,645 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,645 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,645 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,684 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,684 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,685 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,737 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,737 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,737 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,783 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-68810910 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,783 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,783 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: ctest-pod-68810910 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,821 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,821 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,822 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,864 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,864 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,864 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,907 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) ctest-pod-68810910 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,907 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,908 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) ctest-pod-68810910 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,947 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,947 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,947 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-68810910 2025-07-23 15:39:00,987 - DEBUG - [Pod ctest-pod-60596179] Cmd: wget http://10.98.179.150:80/ -O - -T 5 -t 1 , Output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,987 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd wget http://10.98.179.150:80/ -O - -T 5 -t 1 passed 2025-07-23 15:39:00,987 - DEBUG - [Pod Pod ctest-pod-60596179] Cmd output: ctest-pod-14502408 Connecting to 10.98.179.150:80 (10.98.179.150:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-07-23 15:39:00,987 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-68810910': 9, 'ctest-pod-14502408': 1} 2025-07-23 15:39:03,027 - DEBUG - [Pod ctest-pod-48554228] Cmd: ping -c 3 10.47.255.244, Output: PING 10.47.255.244 (10.47.255.244): 56 data bytes 64 bytes from 10.47.255.244: seq=0 ttl=63 time=0.379 ms 64 bytes from 10.47.255.244: seq=1 ttl=63 time=0.068 ms 64 bytes from 10.47.255.244: seq=2 ttl=63 time=0.062 ms --- 10.47.255.244 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.062/0.169/0.379 ms 2025-07-23 15:39:03,027 - INFO - Ping check to IP 10.47.255.244 from pod ctest-pod-48554228 with expectation True passed 2025-07-23 15:39:05,073 - DEBUG - [Pod ctest-pod-00885831] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=0.373 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.080 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.115 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.080/0.189/0.373 ms 2025-07-23 15:39:05,074 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-00885831 with expectation True passed 2025-07-23 15:39:07,116 - DEBUG - [Pod ctest-pod-60596179] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=0.349 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.087 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.074 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.074/0.170/0.349 ms 2025-07-23 15:39:07,117 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-60596179 with expectation True passed 2025-07-23 15:39:07,179 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:39:07,180 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 passed 2025-07-23 15:39:07,180 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:39:07,180 - INFO - wget check of of http://10.47.255.248 from ctest-pod-48554228 passed 2025-07-23 15:39:07,229 - DEBUG - [Pod ctest-pod-48554228] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:39:07,229 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 passed 2025-07-23 15:39:07,229 - DEBUG - [Pod Pod ctest-pod-48554228] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:39:07,229 - INFO - wget check of of http://10.47.255.247 from ctest-pod-48554228 passed 2025-07-23 15:39:12,273 - DEBUG - [Pod ctest-pod-00885831] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2025-07-23 15:39:12,274 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2025-07-23 15:39:12,274 - DEBUG - [Pod Pod ctest-pod-00885831] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2025-07-23 15:39:12,274 - INFO - wget check of of http://10.47.255.247 from ctest-pod-00885831 passed 2025-07-23 15:39:12,312 - DEBUG - [Pod ctest-pod-48597499] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:39:12,312 - DEBUG - [Pod Pod ctest-pod-48597499] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 passed 2025-07-23 15:39:12,312 - DEBUG - [Pod Pod ctest-pod-48597499] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-07-23 15:39:12,312 - INFO - wget check of of http://10.47.255.248 from ctest-pod-48597499 passed 2025-07-23 15:39:12,312 - INFO - Deleting Network Policy : ingress-on-service 2025-07-23 15:39:12,313 - INFO - Deleting service : ctest-nginx-svc-59434346 2025-07-23 15:39:12,840 - DEBUG - No XMPP flaps were noticed during the test 2025-07-23 15:39:12,840 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:12] 2025-07-23 15:39:12,840 - INFO - -------------------------------------------------------------------------------- 2025-07-23 15:39:12,845 - INFO - Deleting pod ctest-new-default-86078139:ctest-pod-68810910 2025-07-23 15:39:12,846 - INFO - Deleting pod ctest-new-default-86078139:ctest-pod-14502408 2025-07-23 15:39:12,847 - INFO - Deleting pod ctest-non-default-23262455:ctest-pod-12527673 2025-07-23 15:39:12,847 - INFO - Deleting pod ctest-non-default-23262455:ctest-pod-84632467 2025-07-23 15:39:12,847 - INFO - Deleting pod ctest-new-default-86078139:ctest-pod-00885831 2025-07-23 15:39:12,848 - INFO - Deleting pod ctest-new-default-86078139:ctest-pod-48554228 2025-07-23 15:39:12,848 - INFO - Deleting pod ctest-non-default-23262455:ctest-pod-60596179 2025-07-23 15:39:12,848 - INFO - Deleting pod ctest-non-default-23262455:ctest-pod-48597499 2025-07-23 15:39:12,997 - WARNING - Pod uuid 8765e13f-ce62-4e50-8918-a7e5be9f42e5 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,005 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,025 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,028 - WARNING - Pod uuid 97aab466-138a-4f1f-bc12-453376e66111 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,028 - WARNING - Pod uuid 0b3d65e6-68c0-49a5-aefa-23a19f7fa41a is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,028 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,029 - WARNING - Pod uuid a1b86db8-f295-4a05-a0c3-5bc74e209daf is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:13,029 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,010 - WARNING - Pod uuid 8765e13f-ce62-4e50-8918-a7e5be9f42e5 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,015 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,039 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,056 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,057 - WARNING - Pod uuid a1b86db8-f295-4a05-a0c3-5bc74e209daf is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,057 - WARNING - Pod uuid 0b3d65e6-68c0-49a5-aefa-23a19f7fa41a is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,057 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:18,057 - DEBUG - Pod 97aab466-138a-4f1f-bc12-453376e66111 is not in agent 10.0.0.254 VM list 2025-07-23 15:39:18,064 - DEBUG - VMI a6552bcc-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:18,064 - INFO - Verified that pod ctest-pod-12527673 is removed in agent 2025-07-23 15:39:23,018 - DEBUG - Pod 8765e13f-ce62-4e50-8918-a7e5be9f42e5 is not in agent 10.0.0.254 VM list 2025-07-23 15:39:23,024 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:23,025 - DEBUG - VMI a201b1ee-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:23,025 - INFO - Verified that pod ctest-pod-68810910 is removed in agent 2025-07-23 15:39:23,045 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:23,073 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:23,074 - DEBUG - Pod 0b3d65e6-68c0-49a5-aefa-23a19f7fa41a is not in agent 10.0.0.254 VM list 2025-07-23 15:39:23,076 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:23,076 - WARNING - Pod uuid a1b86db8-f295-4a05-a0c3-5bc74e209daf is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:23,081 - DEBUG - VMI a8772d1a-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:23,081 - INFO - Verified that pod ctest-pod-84632467 is removed in agent 2025-07-23 15:39:28,031 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:28,053 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:28,088 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:28,089 - DEBUG - Pod a1b86db8-f295-4a05-a0c3-5bc74e209daf is not in agent 10.0.0.254 VM list 2025-07-23 15:39:28,094 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:28,095 - DEBUG - VMI a38fecba-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:28,095 - INFO - Verified that pod ctest-pod-14502408 is removed in agent 2025-07-23 15:39:33,040 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:33,062 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:33,100 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:33,116 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:38,047 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:38,069 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:38,112 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:38,123 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:43,056 - WARNING - Pod uuid b5664ebd-f838-4ebc-8cc2-b36661e967d2 is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:43,076 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:43,121 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:43,134 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:48,063 - DEBUG - Pod b5664ebd-f838-4ebc-8cc2-b36661e967d2 is not in agent 10.0.0.254 VM list 2025-07-23 15:39:48,069 - DEBUG - VMI b220e5a4-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:48,069 - INFO - Verified that pod ctest-pod-48597499 is removed in agent 2025-07-23 15:39:48,085 - WARNING - Pod uuid 327e4e13-1a92-4b19-a616-7cbde969eede is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:48,128 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:48,149 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:53,107 - DEBUG - Pod 327e4e13-1a92-4b19-a616-7cbde969eede is not in agent 10.0.0.254 VM list 2025-07-23 15:39:53,118 - DEBUG - VMI aaa39880-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:53,118 - INFO - Verified that pod ctest-pod-00885831 is removed in agent 2025-07-23 15:39:53,135 - WARNING - Pod uuid c966a941-a84a-480e-9b7e-5bc651f10a8f is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:53,157 - WARNING - Pod uuid 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is still seen in agent 10.0.0.254 VM list 2025-07-23 15:39:58,142 - DEBUG - Pod c966a941-a84a-480e-9b7e-5bc651f10a8f is not in agent 10.0.0.254 VM list 2025-07-23 15:39:58,147 - DEBUG - VMI aed33b68-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:58,147 - INFO - Verified that pod ctest-pod-60596179 is removed in agent 2025-07-23 15:39:58,164 - DEBUG - Pod 6697c5f2-8ae7-4e82-81ec-beacf1b0521d is not in agent 10.0.0.254 VM list 2025-07-23 15:39:58,170 - DEBUG - VMI ade95e6c-67da-11f0-be54-fa163e7cd4f5 is removed from agent 10.0.0.254 2025-07-23 15:39:58,170 - INFO - Verified that pod ctest-pod-48554228 is removed in agent 2025-07-23 15:39:58,170 - INFO - Deleting namespace ctest-non-default-23262455 2025-07-23 15:39:58,180 - DEBUG - Namespace ctest-non-default-23262455 still in k8s 2025-07-23 15:40:00,185 - DEBUG - Namespace ctest-non-default-23262455 still in k8s 2025-07-23 15:40:02,189 - DEBUG - Namespace ctest-non-default-23262455 still in k8s 2025-07-23 15:40:04,196 - DEBUG - Namespace ctest-non-default-23262455 is not in k8s 2025-07-23 15:40:04,196 - DEBUG - No need to check project deletion in contrail-api 2025-07-23 15:40:04,196 - INFO - Deleting namespace ctest-new-default-86078139 2025-07-23 15:40:04,213 - DEBUG - Namespace ctest-new-default-86078139 still in k8s 2025-07-23 15:40:06,220 - DEBUG - Namespace ctest-new-default-86078139 still in k8s 2025-07-23 15:40:08,226 - DEBUG - Namespace ctest-new-default-86078139 still in k8s 2025-07-23 15:40:10,230 - DEBUG - Namespace ctest-new-default-86078139 is not in k8s 2025-07-23 15:40:10,230 - DEBUG - No need to check project deletion in contrail-api