2024-08-26 17:18:57,255 - INFO - Using existing project ['default-domain', 'k8s-default'](8412b264-9fc4-40e3-858e-da58ed210734) 2024-08-26 17:18:57,283 - DEBUG - Namespace ctest-new-default-73915867 not present 2024-08-26 17:18:57,291 - INFO - Created namespace ctest-new-default-73915867 2024-08-26 17:19:00,297 - DEBUG - Namespace ctest-non-default-08224335 not present 2024-08-26 17:19:00,309 - INFO - Created namespace ctest-non-default-08224335 2024-08-26 17:19:03,327 - DEBUG - Pod ctest-pod-00505641 not present 2024-08-26 17:19:03,327 - INFO - Creating Pod ctest-pod-00505641 2024-08-26 17:19:03,356 - DEBUG - Pod : ctest-pod-00505641 UUID is c2951298-4d63-4129-888c-a644a3251fc2 2024-08-26 17:19:03,359 - DEBUG - Pod ctest-pod-60970608 not present 2024-08-26 17:19:03,359 - INFO - Creating Pod ctest-pod-60970608 2024-08-26 17:19:03,392 - DEBUG - Pod : ctest-pod-60970608 UUID is b49d7953-3bdc-4b5e-b815-d3308f3eb417 2024-08-26 17:19:03,394 - DEBUG - Pod ctest-pod-56725915 not present 2024-08-26 17:19:03,395 - INFO - Creating Pod ctest-pod-56725915 2024-08-26 17:19:03,435 - DEBUG - Pod : ctest-pod-56725915 UUID is 4b47b45a-e765-445d-8bd8-76fbc9142604 2024-08-26 17:19:03,438 - DEBUG - Pod ctest-pod-67103512 not present 2024-08-26 17:19:03,439 - INFO - Creating Pod ctest-pod-67103512 2024-08-26 17:19:03,473 - DEBUG - Pod : ctest-pod-67103512 UUID is 97cc0b22-1a93-4e61-8aff-74d8b650d944 2024-08-26 17:19:03,476 - DEBUG - Pod ctest-pod-33642718 not present 2024-08-26 17:19:03,476 - INFO - Creating Pod ctest-pod-33642718 2024-08-26 17:19:03,496 - DEBUG - Pod : ctest-pod-33642718 UUID is 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 2024-08-26 17:19:03,498 - DEBUG - Pod ctest-pod-60660829 not present 2024-08-26 17:19:03,498 - INFO - Creating Pod ctest-pod-60660829 2024-08-26 17:19:03,525 - DEBUG - Pod : ctest-pod-60660829 UUID is bdc2089b-e663-4963-9471-f5ac5a5999ee 2024-08-26 17:19:03,528 - DEBUG - Pod ctest-pod-74674408 not present 2024-08-26 17:19:03,528 - INFO - Creating Pod ctest-pod-74674408 2024-08-26 17:19:03,552 - DEBUG - Pod : ctest-pod-74674408 UUID is fc99b75a-9403-4b40-bb3c-6f7190f0012e 2024-08-26 17:19:03,554 - DEBUG - Pod ctest-pod-79506901 not present 2024-08-26 17:19:03,554 - INFO - Creating Pod ctest-pod-79506901 2024-08-26 17:19:03,586 - DEBUG - Pod : ctest-pod-79506901 UUID is 3720d41e-3eed-43da-aec9-f0e426a789cc 2024-08-26 17:19:03,808 - INFO - Project default-domain:k8s-ctest-new-default-73915867 for Namespace ctest-new-default-73915867 is seen in contrail-api 2024-08-26 17:19:03,809 - DEBUG - Project uuid in Contrail is fd509516-c4e7-48e7-acd0-958449c94631 2024-08-26 17:19:04,004 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NamespaceDatabaseList?namespace_uuid=29607280-2090-4e14-9666-75d1a0260853 2024-08-26 17:19:04,014 - INFO - Namespace ctest-new-default-73915867 with uuid 29607280-2090-4e14-9666-75d1a0260853 found in kube manager 2024-08-26 17:19:04,014 - INFO - Namespace ctest-new-default-73915867 verification passed 2024-08-26 17:19:04,252 - INFO - Project default-domain:k8s-ctest-non-default-08224335 for Namespace ctest-non-default-08224335 is seen in contrail-api 2024-08-26 17:19:04,252 - DEBUG - Project uuid in Contrail is 6d326ec7-a479-4a8a-9b5d-cf49bf393058 2024-08-26 17:19:04,253 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NamespaceDatabaseList?namespace_uuid=7b0cf4d4-a98e-484d-bad7-450ee341d105 2024-08-26 17:19:04,257 - INFO - Namespace ctest-non-default-08224335 with uuid 7b0cf4d4-a98e-484d-bad7-450ee341d105 found in kube manager 2024-08-26 17:19:04,258 - INFO - Namespace ctest-non-default-08224335 verification passed 2024-08-26 17:19:04,264 - DEBUG - Pod ctest-pod-00505641 not in running state.Currently in Pending 2024-08-26 17:19:09,271 - DEBUG - Pod ctest-pod-00505641 not in running state.Currently in Pending 2024-08-26 17:19:14,280 - DEBUG - Pod ctest-pod-00505641 not in running state.Currently in Pending 2024-08-26 17:19:19,288 - INFO - Pod ctest-pod-00505641 is in running state.Got IP 10.47.255.251 2024-08-26 17:19:19,299 - DEBUG - Pod ctest-pod-00505641 has vmi 4b9881c6-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:19,300 - INFO - Verified pod ctest-pod-00505641 in contrail-api 2024-08-26 17:19:19,314 - DEBUG - VMI 4b9881c6-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:19,314 - INFO - Verified Pod ctest-pod-00505641 in agent 10.0.0.60 2024-08-26 17:19:19,315 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=c2951298-4d63-4129-888c-a644a3251fc2 2024-08-26 17:19:19,319 - INFO - Pod ctest-pod-00505641 with uuid c2951298-4d63-4129-888c-a644a3251fc2 found in kube manager 2024-08-26 17:19:19,320 - INFO - Pod ctest-pod-00505641 verification passed 2024-08-26 17:19:19,326 - INFO - Pod ctest-pod-60970608 is in running state.Got IP 10.47.255.250 2024-08-26 17:19:19,385 - DEBUG - Pod ctest-pod-60970608 has vmi 4c12e3c6-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:19,385 - INFO - Verified pod ctest-pod-60970608 in contrail-api 2024-08-26 17:19:19,398 - DEBUG - VMI 4c12e3c6-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:19,398 - INFO - Verified Pod ctest-pod-60970608 in agent 10.0.0.60 2024-08-26 17:19:19,399 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=b49d7953-3bdc-4b5e-b815-d3308f3eb417 2024-08-26 17:19:19,404 - INFO - Pod ctest-pod-60970608 with uuid b49d7953-3bdc-4b5e-b815-d3308f3eb417 found in kube manager 2024-08-26 17:19:19,405 - INFO - Pod ctest-pod-60970608 verification passed 2024-08-26 17:19:19,411 - INFO - Pod ctest-pod-56725915 is in running state.Got IP 10.47.255.249 2024-08-26 17:19:19,423 - DEBUG - Pod ctest-pod-56725915 has vmi 4cc577b6-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:19,423 - INFO - Verified pod ctest-pod-56725915 in contrail-api 2024-08-26 17:19:19,439 - DEBUG - VMI 4cc577b6-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:19,439 - INFO - Verified Pod ctest-pod-56725915 in agent 10.0.0.60 2024-08-26 17:19:19,439 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=4b47b45a-e765-445d-8bd8-76fbc9142604 2024-08-26 17:19:19,445 - INFO - Pod ctest-pod-56725915 with uuid 4b47b45a-e765-445d-8bd8-76fbc9142604 found in kube manager 2024-08-26 17:19:19,445 - INFO - Pod ctest-pod-56725915 verification passed 2024-08-26 17:19:19,454 - INFO - Pod ctest-pod-67103512 is in running state.Got IP 10.47.255.248 2024-08-26 17:19:19,511 - DEBUG - Pod ctest-pod-67103512 has vmi 4d4fd078-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:19,511 - INFO - Verified pod ctest-pod-67103512 in contrail-api 2024-08-26 17:19:19,526 - DEBUG - VMI 4d4fd078-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:19,526 - INFO - Verified Pod ctest-pod-67103512 in agent 10.0.0.60 2024-08-26 17:19:19,526 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=97cc0b22-1a93-4e61-8aff-74d8b650d944 2024-08-26 17:19:19,532 - INFO - Pod ctest-pod-67103512 with uuid 97cc0b22-1a93-4e61-8aff-74d8b650d944 found in kube manager 2024-08-26 17:19:19,532 - INFO - Pod ctest-pod-67103512 verification passed 2024-08-26 17:19:19,539 - INFO - Pod ctest-pod-33642718 is in running state.Got IP 10.47.255.247 2024-08-26 17:19:19,593 - DEBUG - Pod ctest-pod-33642718 has vmi 4de0d21c-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:19,593 - INFO - Verified pod ctest-pod-33642718 in contrail-api 2024-08-26 17:19:19,608 - DEBUG - VMI 4de0d21c-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:19,608 - INFO - Verified Pod ctest-pod-33642718 in agent 10.0.0.60 2024-08-26 17:19:19,608 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 2024-08-26 17:19:19,626 - INFO - Pod ctest-pod-33642718 with uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 found in kube manager 2024-08-26 17:19:19,626 - INFO - Pod ctest-pod-33642718 verification passed 2024-08-26 17:19:19,634 - INFO - Pod ctest-pod-74674408 is in running state.Got IP 10.47.255.245 2024-08-26 17:19:19,646 - DEBUG - Pod ctest-pod-74674408 has vmi 4eea825c-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:19,646 - INFO - Verified pod ctest-pod-74674408 in contrail-api 2024-08-26 17:19:19,659 - DEBUG - VMI 4eea825c-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:19,660 - INFO - Verified Pod ctest-pod-74674408 in agent 10.0.0.60 2024-08-26 17:19:19,660 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=fc99b75a-9403-4b40-bb3c-6f7190f0012e 2024-08-26 17:19:19,665 - INFO - Pod ctest-pod-74674408 with uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e found in kube manager 2024-08-26 17:19:19,665 - INFO - Pod ctest-pod-74674408 verification passed 2024-08-26 17:19:19,670 - DEBUG - Pod ctest-pod-60660829 not in running state.Currently in Pending 2024-08-26 17:19:24,678 - INFO - Pod ctest-pod-60660829 is in running state.Got IP 10.47.255.246 2024-08-26 17:19:24,691 - DEBUG - Pod ctest-pod-60660829 has vmi 4e6cc70e-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:24,691 - INFO - Verified pod ctest-pod-60660829 in contrail-api 2024-08-26 17:19:24,705 - DEBUG - VMI 4e6cc70e-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:24,705 - INFO - Verified Pod ctest-pod-60660829 in agent 10.0.0.60 2024-08-26 17:19:24,706 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=bdc2089b-e663-4963-9471-f5ac5a5999ee 2024-08-26 17:19:24,711 - INFO - Pod ctest-pod-60660829 with uuid bdc2089b-e663-4963-9471-f5ac5a5999ee found in kube manager 2024-08-26 17:19:24,711 - INFO - Pod ctest-pod-60660829 verification passed 2024-08-26 17:19:24,717 - INFO - Pod ctest-pod-79506901 is in running state.Got IP 10.47.255.244 2024-08-26 17:19:24,777 - DEBUG - Pod ctest-pod-79506901 has vmi 4f5fadc0-63cf-11ef-820a-fa163efab78c 2024-08-26 17:19:24,777 - INFO - Verified pod ctest-pod-79506901 in contrail-api 2024-08-26 17:19:24,794 - DEBUG - VMI 4f5fadc0-63cf-11ef-820a-fa163efab78c is active in agent 10.0.0.60 2024-08-26 17:19:24,794 - INFO - Verified Pod ctest-pod-79506901 in agent 10.0.0.60 2024-08-26 17:19:24,795 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=3720d41e-3eed-43da-aec9-f0e426a789cc 2024-08-26 17:19:24,812 - INFO - Pod ctest-pod-79506901 with uuid 3720d41e-3eed-43da-aec9-f0e426a789cc found in kube manager 2024-08-26 17:19:24,813 - INFO - Pod ctest-pod-79506901 verification passed 2024-08-26 17:19:24,813 - INFO - ================================================================================ 2024-08-26 17:19:24,814 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-08-26 17:19:24,815 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-08-26 17:19:25,135 - DEBUG - Nothing to compare xmpp stats {'10.0.0.60': {'10.0.0.42': '0'}} with 2024-08-26 17:19:25,135 - INFO - Initial checks done. Running the testcase now 2024-08-26 17:19:25,135 - INFO - 2024-08-26 17:19:25,204 - DEBUG - [Pod ctest-pod-00505641] Cmd: echo ctest-pod-00505641 > /usr/share/nginx/html/index.html, Output: 2024-08-26 17:19:25,270 - DEBUG - [Pod ctest-pod-60970608] Cmd: echo ctest-pod-60970608 > /usr/share/nginx/html/index.html, Output: 2024-08-26 17:19:25,332 - DEBUG - [Pod ctest-pod-56725915] Cmd: echo ctest-pod-56725915 > /usr/share/nginx/html/index.html, Output: 2024-08-26 17:19:25,388 - DEBUG - [Pod ctest-pod-67103512] Cmd: echo ctest-pod-67103512 > /usr/share/nginx/html/index.html, Output: 2024-08-26 17:19:25,391 - DEBUG - Service ctest-nginx-svc-30578732 not present 2024-08-26 17:19:25,391 - INFO - Creating service ctest-nginx-svc-30578732 2024-08-26 17:19:25,406 - INFO - Created Service ctest-nginx-svc-30578732 2024-08-26 17:19:25,421 - WARNING - Service UUID 6fe70cde-3cc3-41aa-88ba-a69ca33c0ba7 not yet found in contrail-api 2024-08-26 17:19:26,432 - INFO - Validated that Service ctest-nginx-svc-30578732 is seen in contrail-api 2024-08-26 17:19:26,433 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_LoadbalancerDatabaseList?x=6fe70cde-3cc3-41aa-88ba-a69ca33c0ba7 2024-08-26 17:19:26,440 - INFO - Service ctest-nginx-svc-30578732 with uuid 6fe70cde-3cc3-41aa-88ba-a69ca33c0ba7 found in kube manager 2024-08-26 17:19:26,440 - INFO - Service ctest-nginx-svc-30578732 verification passed 2024-08-26 17:19:27,501 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-60970608 2024-08-26 17:19:27,501 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:19:27,501 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-60970608 2024-08-26 17:19:27,571 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-00505641 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:19:27,571 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:19:27,571 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: ctest-pod-00505641 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:19:27,571 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-00505641': 1, 'ctest-pod-60970608': 1} 2024-08-26 17:19:27,620 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:19:27,620 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:19:27,620 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:19:27,685 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-00505641 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:19:27,685 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:19:27,685 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: ctest-pod-00505641 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:19:27,685 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-00505641': 1, 'ctest-pod-60970608': 1} 2024-08-26 17:19:27,689 - DEBUG - Network policy ingress-on-service not present 2024-08-26 17:19:27,689 - INFO - Creating Network Policy ingress-on-service 2024-08-26 17:19:27,710 - INFO - Network policy found in k8s 2024-08-26 17:19:27,710 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=08b2e153-619c-4d45-911e-3bdc22b2a381 2024-08-26 17:19:27,716 - INFO - Network Policy ingress-on-service with uuid 08b2e153-619c-4d45-911e-3bdc22b2a381 found in kube manager 2024-08-26 17:19:27,770 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-08-26 17:19:27,771 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-08-26 17:19:27,771 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=08b2e153-619c-4d45-911e-3bdc22b2a381 2024-08-26 17:19:27,811 - WARNING - Network policy with name ingress-on-service not found in agent 2024-08-26 17:19:29,811 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=08b2e153-619c-4d45-911e-3bdc22b2a381 2024-08-26 17:19:29,848 - INFO - Network policy with name ingress-on-service found in agent 2024-08-26 17:19:29,848 - INFO - Network Policy ingress-on-service verification passed 2024-08-26 17:19:34,903 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:34,903 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:34,903 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:39,961 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:39,962 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:39,962 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:45,036 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:45,036 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:45,036 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:50,101 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:50,101 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:50,101 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:55,160 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:55,160 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:19:55,160 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:00,215 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:00,215 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:00,215 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:05,276 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:05,277 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:05,277 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:10,354 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:10,354 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:10,354 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:15,426 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:15,426 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:15,427 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:20,489 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:20,489 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:20,489 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:20,489 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-00505641': 0, 'ctest-pod-60970608': 0} 2024-08-26 17:20:20,564 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:20:20,564 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:20:20,564 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:20:20,627 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-00505641 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:20:20,627 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:20:20,627 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: ctest-pod-00505641 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:20:20,627 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-00505641': 1, 'ctest-pod-60970608': 1} 2024-08-26 17:20:25,678 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:25,678 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:25,678 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:30,734 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:30,734 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:30,734 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:34,064 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2024-08-26 17:20:44,407 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:44,407 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:44,407 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:49,476 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:49,476 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:49,476 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:54,558 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:54,558 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:54,558 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:59,634 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:59,634 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:20:59,634 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:04,696 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:04,696 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:04,697 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:09,768 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:09,769 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:09,769 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:14,843 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:14,843 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:14,843 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:19,906 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:19,906 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:19,906 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:19,906 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-00505641': 0, 'ctest-pod-60970608': 0} 2024-08-26 17:21:21,977 - DEBUG - [Pod ctest-pod-60660829] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.188 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.072 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.088 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.072/0.449/1.188 ms 2024-08-26 17:21:21,977 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-60660829 with expectation True passed 2024-08-26 17:21:24,041 - DEBUG - [Pod ctest-pod-33642718] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.017 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.069 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.079 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.069/0.388/1.017 ms 2024-08-26 17:21:24,041 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-33642718 with expectation True passed 2024-08-26 17:21:26,110 - DEBUG - [Pod ctest-pod-74674408] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.330 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.074 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.088 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.074/0.497/1.330 ms 2024-08-26 17:21:26,110 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-74674408 with expectation True passed 2024-08-26 17:21:26,177 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:21:26,177 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-08-26 17:21:26,177 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:21:26,177 - INFO - wget check of of http://10.47.255.251 from ctest-pod-60660829 passed 2024-08-26 17:21:26,249 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:21:26,249 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-08-26 17:21:26,250 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:21:26,250 - INFO - wget check of of http://10.47.255.250 from ctest-pod-60660829 passed 2024-08-26 17:21:31,300 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-26 17:21:31,300 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-26 17:21:31,300 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-26 17:21:31,300 - INFO - wget check of of http://10.47.255.250 from ctest-pod-33642718 passed 2024-08-26 17:21:36,370 - DEBUG - [Pod ctest-pod-79506901] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-08-26 17:21:36,370 - DEBUG - [Pod Pod ctest-pod-79506901] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-08-26 17:21:36,370 - DEBUG - [Pod Pod ctest-pod-79506901] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-08-26 17:21:36,370 - INFO - wget check of of http://10.47.255.251 from ctest-pod-79506901 passed 2024-08-26 17:21:36,379 - INFO - Updating Network Policy ingress-on-service 2024-08-26 17:21:41,456 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:41,456 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:41,457 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:46,516 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:46,516 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:46,516 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:51,577 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:51,577 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:51,577 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:56,631 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:56,631 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:21:56,631 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:01,683 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:01,684 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:01,684 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:06,733 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:06,733 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:06,733 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:11,814 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:11,814 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:11,814 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:16,882 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:16,882 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:16,882 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:21,945 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:21,945 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:21,945 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:27,022 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:27,022 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:27,022 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) wget: download timed out 2024-08-26 17:22:27,022 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-00505641': 0, 'ctest-pod-60970608': 0} 2024-08-26 17:22:27,092 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:22:27,092 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:22:27,092 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:22:27,159 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-00505641 2024-08-26 17:22:27,160 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:22:27,160 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-00505641 2024-08-26 17:22:27,160 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-00505641': 1, 'ctest-pod-60970608': 1} 2024-08-26 17:22:27,227 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:22:27,227 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:22:27,227 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: ctest-pod-60970608 Connecting to 10.103.56.41:80 (10.103.56.41:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:22:27,292 - DEBUG - [Pod ctest-pod-74674408] Cmd: wget http://10.103.56.41:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.56.41:80 (10.103.56.41:80) ctest-pod-00505641 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:22:27,292 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd wget http://10.103.56.41:80/ -O - -T 5 -t 1 passed 2024-08-26 17:22:27,292 - DEBUG - [Pod Pod ctest-pod-74674408] Cmd output: Connecting to 10.103.56.41:80 (10.103.56.41:80) ctest-pod-00505641 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-26 17:22:27,292 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-00505641': 1, 'ctest-pod-60970608': 1} 2024-08-26 17:22:29,357 - DEBUG - [Pod ctest-pod-60660829] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.130 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.069 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.078 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.069/0.425/1.130 ms 2024-08-26 17:22:29,357 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-60660829 with expectation True passed 2024-08-26 17:22:31,423 - DEBUG - [Pod ctest-pod-33642718] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.458 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.088 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.075 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.075/0.207/0.458 ms 2024-08-26 17:22:31,423 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-33642718 with expectation True passed 2024-08-26 17:22:33,478 - DEBUG - [Pod ctest-pod-74674408] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.425 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.070 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.070 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.070/0.188/0.425 ms 2024-08-26 17:22:33,478 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-74674408 with expectation True passed 2024-08-26 17:22:33,526 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:22:33,527 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-08-26 17:22:33,527 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:22:33,527 - INFO - wget check of of http://10.47.255.251 from ctest-pod-60660829 passed 2024-08-26 17:22:33,582 - DEBUG - [Pod ctest-pod-60660829] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:22:33,582 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-08-26 17:22:33,582 - DEBUG - [Pod Pod ctest-pod-60660829] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:22:33,582 - INFO - wget check of of http://10.47.255.250 from ctest-pod-60660829 passed 2024-08-26 17:22:38,650 - DEBUG - [Pod ctest-pod-33642718] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-26 17:22:38,650 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-26 17:22:38,650 - DEBUG - [Pod Pod ctest-pod-33642718] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-26 17:22:38,650 - INFO - wget check of of http://10.47.255.250 from ctest-pod-33642718 passed 2024-08-26 17:22:38,712 - DEBUG - [Pod ctest-pod-79506901] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:22:38,712 - DEBUG - [Pod Pod ctest-pod-79506901] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-08-26 17:22:38,712 - DEBUG - [Pod Pod ctest-pod-79506901] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-26 17:22:38,712 - INFO - wget check of of http://10.47.255.251 from ctest-pod-79506901 passed 2024-08-26 17:22:38,712 - INFO - Deleting Network Policy : ingress-on-service 2024-08-26 17:22:38,713 - INFO - Deleting service : ctest-nginx-svc-30578732 2024-08-26 17:22:38,938 - DEBUG - No XMPP flaps were noticed during the test 2024-08-26 17:22:38,938 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:14] 2024-08-26 17:22:38,938 - INFO - -------------------------------------------------------------------------------- 2024-08-26 17:22:38,944 - INFO - Deleting pod ctest-new-default-73915867:ctest-pod-00505641 2024-08-26 17:22:38,945 - INFO - Deleting pod ctest-new-default-73915867:ctest-pod-60970608 2024-08-26 17:22:38,946 - INFO - Deleting pod ctest-non-default-08224335:ctest-pod-56725915 2024-08-26 17:22:38,946 - INFO - Deleting pod ctest-non-default-08224335:ctest-pod-67103512 2024-08-26 17:22:38,947 - INFO - Deleting pod ctest-new-default-73915867:ctest-pod-33642718 2024-08-26 17:22:38,947 - INFO - Deleting pod ctest-new-default-73915867:ctest-pod-60660829 2024-08-26 17:22:38,948 - INFO - Deleting pod ctest-non-default-08224335:ctest-pod-74674408 2024-08-26 17:22:38,949 - INFO - Deleting pod ctest-non-default-08224335:ctest-pod-79506901 2024-08-26 17:22:39,017 - WARNING - Pod uuid c2951298-4d63-4129-888c-a644a3251fc2 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,159 - WARNING - Pod uuid b49d7953-3bdc-4b5e-b815-d3308f3eb417 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,160 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,182 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,184 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,184 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,185 - WARNING - Pod uuid 4b47b45a-e765-445d-8bd8-76fbc9142604 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:39,185 - WARNING - Pod uuid 97cc0b22-1a93-4e61-8aff-74d8b650d944 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:44,030 - DEBUG - Pod c2951298-4d63-4129-888c-a644a3251fc2 is not in agent 10.0.0.60 VM list 2024-08-26 17:22:44,038 - DEBUG - VMI 4b9881c6-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:22:44,038 - INFO - Verified that pod ctest-pod-00505641 is removed in agent 2024-08-26 17:22:44,175 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:44,175 - DEBUG - Pod b49d7953-3bdc-4b5e-b815-d3308f3eb417 is not in agent 10.0.0.60 VM list 2024-08-26 17:22:44,183 - DEBUG - VMI 4c12e3c6-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:22:44,183 - INFO - Verified that pod ctest-pod-60970608 is removed in agent 2024-08-26 17:22:44,210 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:44,211 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:44,213 - DEBUG - Pod 4b47b45a-e765-445d-8bd8-76fbc9142604 is not in agent 10.0.0.60 VM list 2024-08-26 17:22:44,214 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:44,215 - DEBUG - Pod 97cc0b22-1a93-4e61-8aff-74d8b650d944 is not in agent 10.0.0.60 VM list 2024-08-26 17:22:44,226 - DEBUG - VMI 4d4fd078-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:22:44,226 - INFO - Verified that pod ctest-pod-67103512 is removed in agent 2024-08-26 17:22:44,227 - DEBUG - VMI 4cc577b6-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:22:44,227 - INFO - Verified that pod ctest-pod-56725915 is removed in agent 2024-08-26 17:22:49,184 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:49,239 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:49,239 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:49,240 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:54,192 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:54,255 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:54,257 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:54,258 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:59,202 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:59,275 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:59,277 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:22:59,278 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:04,211 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:04,291 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:04,293 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:04,293 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:09,237 - WARNING - Pod uuid fc99b75a-9403-4b40-bb3c-6f7190f0012e is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:09,316 - WARNING - Pod uuid bdc2089b-e663-4963-9471-f5ac5a5999ee is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:09,317 - WARNING - Pod uuid 3720d41e-3eed-43da-aec9-f0e426a789cc is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:09,317 - WARNING - Pod uuid 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is still seen in agent 10.0.0.60 VM list 2024-08-26 17:23:14,246 - DEBUG - Pod fc99b75a-9403-4b40-bb3c-6f7190f0012e is not in agent 10.0.0.60 VM list 2024-08-26 17:23:14,254 - DEBUG - VMI 4eea825c-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:23:14,254 - INFO - Verified that pod ctest-pod-74674408 is removed in agent 2024-08-26 17:23:14,339 - DEBUG - Pod bdc2089b-e663-4963-9471-f5ac5a5999ee is not in agent 10.0.0.60 VM list 2024-08-26 17:23:14,343 - DEBUG - Pod 3720d41e-3eed-43da-aec9-f0e426a789cc is not in agent 10.0.0.60 VM list 2024-08-26 17:23:14,344 - DEBUG - Pod 8e0e59d2-a9aa-47c6-96e5-b1074f0ff9b6 is not in agent 10.0.0.60 VM list 2024-08-26 17:23:14,359 - DEBUG - VMI 4e6cc70e-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:23:14,359 - INFO - Verified that pod ctest-pod-60660829 is removed in agent 2024-08-26 17:23:14,361 - DEBUG - VMI 4de0d21c-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:23:14,361 - INFO - Verified that pod ctest-pod-33642718 is removed in agent 2024-08-26 17:23:14,362 - DEBUG - VMI 4f5fadc0-63cf-11ef-820a-fa163efab78c is removed from agent 10.0.0.60 2024-08-26 17:23:14,362 - INFO - Verified that pod ctest-pod-79506901 is removed in agent 2024-08-26 17:23:14,362 - INFO - Deleting namespace ctest-non-default-08224335 2024-08-26 17:23:14,386 - DEBUG - Namespace ctest-non-default-08224335 still in k8s 2024-08-26 17:23:16,392 - DEBUG - Namespace ctest-non-default-08224335 still in k8s 2024-08-26 17:23:18,402 - DEBUG - Namespace ctest-non-default-08224335 still in k8s 2024-08-26 17:23:20,407 - DEBUG - Namespace ctest-non-default-08224335 is not in k8s 2024-08-26 17:23:20,407 - DEBUG - No need to check project deletion in contrail-api 2024-08-26 17:23:20,407 - INFO - Deleting namespace ctest-new-default-73915867 2024-08-26 17:23:20,420 - DEBUG - Namespace ctest-new-default-73915867 still in k8s 2024-08-26 17:23:22,427 - DEBUG - Namespace ctest-new-default-73915867 still in k8s 2024-08-26 17:23:24,435 - DEBUG - Namespace ctest-new-default-73915867 still in k8s 2024-08-26 17:23:26,440 - DEBUG - Namespace ctest-new-default-73915867 is not in k8s 2024-08-26 17:23:26,440 - DEBUG - No need to check project deletion in contrail-api