2025-03-28 22:23:47,117 - INFO - Using existing project ['default-domain', 'k8s-default'](f7522d62-6bef-4fa6-9cf2-71384ceb3fe2) 2025-03-28 22:23:47,140 - DEBUG - Namespace ctest-new-default-20637346 not present 2025-03-28 22:23:47,149 - INFO - Created namespace ctest-new-default-20637346 2025-03-28 22:23:50,154 - DEBUG - Namespace ctest-non-default-05997853 not present 2025-03-28 22:23:50,168 - INFO - Created namespace ctest-non-default-05997853 2025-03-28 22:23:53,187 - DEBUG - Pod ctest-pod-62058388 not present 2025-03-28 22:23:53,187 - INFO - Creating Pod ctest-pod-62058388 2025-03-28 22:23:53,207 - DEBUG - Pod : ctest-pod-62058388 UUID is 64b25dab-872b-4dab-9b60-554bcb1e7402 2025-03-28 22:23:53,224 - DEBUG - Pod ctest-pod-86807875 not present 2025-03-28 22:23:53,225 - INFO - Creating Pod ctest-pod-86807875 2025-03-28 22:23:53,262 - DEBUG - Pod : ctest-pod-86807875 UUID is 091059e8-53d3-4a47-b2b7-7d1c27276a92 2025-03-28 22:23:53,267 - DEBUG - Pod ctest-pod-72014971 not present 2025-03-28 22:23:53,268 - INFO - Creating Pod ctest-pod-72014971 2025-03-28 22:23:53,289 - DEBUG - Pod : ctest-pod-72014971 UUID is 7ba28042-ca7d-4833-ae19-b7891762cd22 2025-03-28 22:23:53,292 - DEBUG - Pod ctest-pod-41237285 not present 2025-03-28 22:23:53,292 - INFO - Creating Pod ctest-pod-41237285 2025-03-28 22:23:53,311 - DEBUG - Pod : ctest-pod-41237285 UUID is bd417630-b197-4e07-9e35-7a8127a2610f 2025-03-28 22:23:53,314 - DEBUG - Pod ctest-pod-23078103 not present 2025-03-28 22:23:53,314 - INFO - Creating Pod ctest-pod-23078103 2025-03-28 22:23:53,353 - DEBUG - Pod : ctest-pod-23078103 UUID is f7a0187a-2ad5-4b38-8ba2-2685740e1510 2025-03-28 22:23:53,355 - DEBUG - Pod ctest-pod-37649715 not present 2025-03-28 22:23:53,356 - INFO - Creating Pod ctest-pod-37649715 2025-03-28 22:23:53,372 - DEBUG - Pod : ctest-pod-37649715 UUID is 9647793b-be0e-48e6-ac5d-418bd8b17050 2025-03-28 22:23:53,374 - DEBUG - Pod ctest-pod-33369321 not present 2025-03-28 22:23:53,374 - INFO - Creating Pod ctest-pod-33369321 2025-03-28 22:23:53,395 - DEBUG - Pod : ctest-pod-33369321 UUID is 59972a4f-446c-4b82-bdf2-f379df51b269 2025-03-28 22:23:53,401 - DEBUG - Pod ctest-pod-49877151 not present 2025-03-28 22:23:53,401 - INFO - Creating Pod ctest-pod-49877151 2025-03-28 22:23:53,422 - DEBUG - Pod : ctest-pod-49877151 UUID is 7632dfa5-7f71-4193-a4ce-48105d9f9d36 2025-03-28 22:23:53,609 - INFO - Project default-domain:k8s-ctest-new-default-20637346 for Namespace ctest-new-default-20637346 is seen in contrail-api 2025-03-28 22:23:53,609 - DEBUG - Project uuid in Contrail is 35880c7b-bc03-4c98-a622-2dcd70db1a2e 2025-03-28 22:23:53,739 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NamespaceDatabaseList?namespace_uuid=d979e8c3-3d3c-4f9f-9de0-b9ab38a0f0db 2025-03-28 22:23:53,744 - INFO - Namespace ctest-new-default-20637346 with uuid d979e8c3-3d3c-4f9f-9de0-b9ab38a0f0db found in kube manager 2025-03-28 22:23:53,744 - INFO - Namespace ctest-new-default-20637346 verification passed 2025-03-28 22:23:53,976 - INFO - Project default-domain:k8s-ctest-non-default-05997853 for Namespace ctest-non-default-05997853 is seen in contrail-api 2025-03-28 22:23:53,976 - DEBUG - Project uuid in Contrail is 3ae9d9b5-94a7-4f46-a98a-c3c56ea2edd3 2025-03-28 22:23:53,976 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NamespaceDatabaseList?namespace_uuid=0003fdaf-eba3-4d9e-a0c0-213285884764 2025-03-28 22:23:53,981 - INFO - Namespace ctest-non-default-05997853 with uuid 0003fdaf-eba3-4d9e-a0c0-213285884764 found in kube manager 2025-03-28 22:23:53,981 - INFO - Namespace ctest-non-default-05997853 verification passed 2025-03-28 22:23:53,986 - DEBUG - Pod ctest-pod-62058388 not in running state.Currently in Pending 2025-03-28 22:23:58,993 - DEBUG - Pod ctest-pod-62058388 not in running state.Currently in Pending 2025-03-28 22:24:04,000 - DEBUG - Pod ctest-pod-62058388 not in running state.Currently in Pending 2025-03-28 22:24:09,006 - INFO - Pod ctest-pod-62058388 is in running state.Got IP 10.47.255.251 2025-03-28 22:24:09,020 - DEBUG - Pod ctest-pod-62058388 has vmi 55b6e4ca-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:09,020 - INFO - Verified pod ctest-pod-62058388 in contrail-api 2025-03-28 22:24:09,037 - DEBUG - VMI 55b6e4ca-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:09,037 - INFO - Verified Pod ctest-pod-62058388 in agent 10.0.0.24 2025-03-28 22:24:09,037 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=64b25dab-872b-4dab-9b60-554bcb1e7402 2025-03-28 22:24:09,048 - INFO - Pod ctest-pod-62058388 with uuid 64b25dab-872b-4dab-9b60-554bcb1e7402 found in kube manager 2025-03-28 22:24:09,048 - INFO - Pod ctest-pod-62058388 verification passed 2025-03-28 22:24:09,053 - INFO - Pod ctest-pod-86807875 is in running state.Got IP 10.47.255.250 2025-03-28 22:24:09,108 - DEBUG - Pod ctest-pod-86807875 has vmi 563a600c-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:09,109 - INFO - Verified pod ctest-pod-86807875 in contrail-api 2025-03-28 22:24:09,124 - DEBUG - VMI 563a600c-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:09,125 - INFO - Verified Pod ctest-pod-86807875 in agent 10.0.0.24 2025-03-28 22:24:09,125 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=091059e8-53d3-4a47-b2b7-7d1c27276a92 2025-03-28 22:24:09,130 - INFO - Pod ctest-pod-86807875 with uuid 091059e8-53d3-4a47-b2b7-7d1c27276a92 found in kube manager 2025-03-28 22:24:09,130 - INFO - Pod ctest-pod-86807875 verification passed 2025-03-28 22:24:09,137 - INFO - Pod ctest-pod-72014971 is in running state.Got IP 10.47.255.249 2025-03-28 22:24:09,155 - DEBUG - Pod ctest-pod-72014971 has vmi 56e35720-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:09,155 - INFO - Verified pod ctest-pod-72014971 in contrail-api 2025-03-28 22:24:09,172 - DEBUG - VMI 56e35720-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:09,173 - INFO - Verified Pod ctest-pod-72014971 in agent 10.0.0.24 2025-03-28 22:24:09,173 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=7ba28042-ca7d-4833-ae19-b7891762cd22 2025-03-28 22:24:09,177 - INFO - Pod ctest-pod-72014971 with uuid 7ba28042-ca7d-4833-ae19-b7891762cd22 found in kube manager 2025-03-28 22:24:09,177 - INFO - Pod ctest-pod-72014971 verification passed 2025-03-28 22:24:09,183 - DEBUG - Pod ctest-pod-41237285 not in running state.Currently in Pending 2025-03-28 22:24:14,188 - INFO - Pod ctest-pod-41237285 is in running state.Got IP 10.47.255.248 2025-03-28 22:24:14,248 - DEBUG - Pod ctest-pod-41237285 has vmi 57650b4e-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:14,249 - INFO - Verified pod ctest-pod-41237285 in contrail-api 2025-03-28 22:24:14,262 - DEBUG - VMI 57650b4e-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:14,262 - INFO - Verified Pod ctest-pod-41237285 in agent 10.0.0.24 2025-03-28 22:24:14,262 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=bd417630-b197-4e07-9e35-7a8127a2610f 2025-03-28 22:24:14,267 - INFO - Pod ctest-pod-41237285 with uuid bd417630-b197-4e07-9e35-7a8127a2610f found in kube manager 2025-03-28 22:24:14,267 - INFO - Pod ctest-pod-41237285 verification passed 2025-03-28 22:24:14,272 - INFO - Pod ctest-pod-23078103 is in running state.Got IP 10.47.255.247 2025-03-28 22:24:14,284 - DEBUG - Pod ctest-pod-23078103 has vmi 57ed7a1a-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:14,284 - INFO - Verified pod ctest-pod-23078103 in contrail-api 2025-03-28 22:24:14,297 - DEBUG - VMI 57ed7a1a-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:14,298 - INFO - Verified Pod ctest-pod-23078103 in agent 10.0.0.24 2025-03-28 22:24:14,298 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=f7a0187a-2ad5-4b38-8ba2-2685740e1510 2025-03-28 22:24:14,302 - INFO - Pod ctest-pod-23078103 with uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 found in kube manager 2025-03-28 22:24:14,302 - INFO - Pod ctest-pod-23078103 verification passed 2025-03-28 22:24:14,307 - INFO - Pod ctest-pod-33369321 is in running state.Got IP 10.47.255.245 2025-03-28 22:24:14,365 - DEBUG - Pod ctest-pod-33369321 has vmi 58dd0404-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:14,365 - INFO - Verified pod ctest-pod-33369321 in contrail-api 2025-03-28 22:24:14,382 - DEBUG - VMI 58dd0404-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:14,382 - INFO - Verified Pod ctest-pod-33369321 in agent 10.0.0.24 2025-03-28 22:24:14,382 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=59972a4f-446c-4b82-bdf2-f379df51b269 2025-03-28 22:24:14,387 - INFO - Pod ctest-pod-33369321 with uuid 59972a4f-446c-4b82-bdf2-f379df51b269 found in kube manager 2025-03-28 22:24:14,388 - INFO - Pod ctest-pod-33369321 verification passed 2025-03-28 22:24:14,394 - INFO - Pod ctest-pod-37649715 is in running state.Got IP 10.47.255.246 2025-03-28 22:24:14,448 - DEBUG - Pod ctest-pod-37649715 has vmi 5863d0d4-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:14,449 - INFO - Verified pod ctest-pod-37649715 in contrail-api 2025-03-28 22:24:14,462 - DEBUG - VMI 5863d0d4-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:14,462 - INFO - Verified Pod ctest-pod-37649715 in agent 10.0.0.24 2025-03-28 22:24:14,462 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=9647793b-be0e-48e6-ac5d-418bd8b17050 2025-03-28 22:24:14,466 - INFO - Pod ctest-pod-37649715 with uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 found in kube manager 2025-03-28 22:24:14,466 - INFO - Pod ctest-pod-37649715 verification passed 2025-03-28 22:24:14,471 - INFO - Pod ctest-pod-49877151 is in running state.Got IP 10.47.255.244 2025-03-28 22:24:14,489 - DEBUG - Pod ctest-pod-49877151 has vmi 595913aa-0c23-11f0-ac62-fa163e221e23 2025-03-28 22:24:14,489 - INFO - Verified pod ctest-pod-49877151 in contrail-api 2025-03-28 22:24:14,502 - DEBUG - VMI 595913aa-0c23-11f0-ac62-fa163e221e23 is active in agent 10.0.0.24 2025-03-28 22:24:14,503 - INFO - Verified Pod ctest-pod-49877151 in agent 10.0.0.24 2025-03-28 22:24:14,503 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=7632dfa5-7f71-4193-a4ce-48105d9f9d36 2025-03-28 22:24:14,507 - INFO - Pod ctest-pod-49877151 with uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 found in kube manager 2025-03-28 22:24:14,507 - INFO - Pod ctest-pod-49877151 verification passed 2025-03-28 22:24:14,507 - INFO - ================================================================================ 2025-03-28 22:24:14,507 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2025-03-28 22:24:14,507 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2025-03-28 22:24:14,737 - DEBUG - Nothing to compare xmpp stats {'10.0.0.24': {'10.0.0.22': '0'}} with 2025-03-28 22:24:14,737 - INFO - Initial checks done. Running the testcase now 2025-03-28 22:24:14,737 - INFO - 2025-03-28 22:24:14,792 - DEBUG - [Pod ctest-pod-62058388] Cmd: echo ctest-pod-62058388 > /usr/share/nginx/html/index.html, Output: 2025-03-28 22:24:14,851 - DEBUG - [Pod ctest-pod-86807875] Cmd: echo ctest-pod-86807875 > /usr/share/nginx/html/index.html, Output: 2025-03-28 22:24:14,937 - DEBUG - [Pod ctest-pod-72014971] Cmd: echo ctest-pod-72014971 > /usr/share/nginx/html/index.html, Output: 2025-03-28 22:24:15,011 - DEBUG - [Pod ctest-pod-41237285] Cmd: echo ctest-pod-41237285 > /usr/share/nginx/html/index.html, Output: 2025-03-28 22:24:15,015 - DEBUG - Service ctest-nginx-svc-94824018 not present 2025-03-28 22:24:15,015 - INFO - Creating service ctest-nginx-svc-94824018 2025-03-28 22:24:15,033 - INFO - Created Service ctest-nginx-svc-94824018 2025-03-28 22:24:15,047 - WARNING - Service UUID 787230f7-d36a-460e-a64c-3b2654be9506 not yet found in contrail-api 2025-03-28 22:24:16,069 - INFO - Validated that Service ctest-nginx-svc-94824018 is seen in contrail-api 2025-03-28 22:24:16,069 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=787230f7-d36a-460e-a64c-3b2654be9506 2025-03-28 22:24:16,075 - INFO - Service ctest-nginx-svc-94824018 with uuid 787230f7-d36a-460e-a64c-3b2654be9506 found in kube manager 2025-03-28 22:24:16,075 - INFO - Service ctest-nginx-svc-94824018 verification passed 2025-03-28 22:24:17,135 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-62058388 2025-03-28 22:24:17,135 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:24:17,135 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-62058388 2025-03-28 22:24:17,208 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-86807875 2025-03-28 22:24:17,208 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:24:17,208 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-86807875 2025-03-28 22:24:17,208 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-62058388': 1, 'ctest-pod-86807875': 1} 2025-03-28 22:24:17,277 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-62058388 2025-03-28 22:24:17,277 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:24:17,277 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-62058388 2025-03-28 22:24:17,353 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:24:17,353 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:24:17,353 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:24:17,353 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-62058388': 1, 'ctest-pod-86807875': 1} 2025-03-28 22:24:17,356 - DEBUG - Network policy ingress-on-service not present 2025-03-28 22:24:17,357 - INFO - Creating Network Policy ingress-on-service 2025-03-28 22:24:17,431 - INFO - Network policy found in k8s 2025-03-28 22:24:17,431 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=533a25d3-53da-4c92-91a3-c083f420fdce 2025-03-28 22:24:17,435 - INFO - Network Policy ingress-on-service with uuid 533a25d3-53da-4c92-91a3-c083f420fdce found in kube manager 2025-03-28 22:24:17,467 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2025-03-28 22:24:17,467 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2025-03-28 22:24:17,468 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=533a25d3-53da-4c92-91a3-c083f420fdce 2025-03-28 22:24:17,497 - WARNING - Network policy with name ingress-on-service not found in agent 2025-03-28 22:24:19,498 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=533a25d3-53da-4c92-91a3-c083f420fdce 2025-03-28 22:24:19,531 - INFO - Network policy with name ingress-on-service found in agent 2025-03-28 22:24:19,531 - INFO - Network Policy ingress-on-service verification passed 2025-03-28 22:24:24,592 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:24,592 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:24,592 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:29,666 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:29,666 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:29,666 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:34,717 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:34,718 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:34,718 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:39,793 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:39,793 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:39,793 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:44,857 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:44,857 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:44,857 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:48,725 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2025-03-28 22:24:58,800 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:58,800 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:24:58,800 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:03,872 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:03,872 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:03,872 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:08,941 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:08,941 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:08,941 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:14,010 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:14,010 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:14,010 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:19,092 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:19,092 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:19,092 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:19,092 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-62058388': 0, 'ctest-pod-86807875': 0} 2025-03-28 22:25:19,160 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: ctest-pod-62058388 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:25:19,161 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:25:19,161 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: ctest-pod-62058388 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:25:19,235 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:25:19,235 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:25:19,235 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:25:19,235 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-62058388': 1, 'ctest-pod-86807875': 1} 2025-03-28 22:25:24,316 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:24,317 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:24,317 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:29,398 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:29,398 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:29,398 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:34,455 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:34,455 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:34,455 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:39,515 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:39,515 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:39,515 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:44,586 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:44,586 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:44,586 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:49,655 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:49,655 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:49,655 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:54,729 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:54,729 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:54,729 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:59,797 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:59,797 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:25:59,797 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:04,881 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:04,881 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:04,881 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:09,940 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:09,941 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:09,941 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:09,941 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-62058388': 0, 'ctest-pod-86807875': 0} 2025-03-28 22:26:12,005 - DEBUG - [Pod ctest-pod-37649715] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.428 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.083 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.106 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.083/0.539/1.428 ms 2025-03-28 22:26:12,006 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-37649715 with expectation True passed 2025-03-28 22:26:14,086 - DEBUG - [Pod ctest-pod-23078103] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.106 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.080 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.080 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.080/0.422/1.106 ms 2025-03-28 22:26:14,086 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-23078103 with expectation True passed 2025-03-28 22:26:16,157 - DEBUG - [Pod ctest-pod-33369321] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.341 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.078 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.084 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.078/0.167/0.341 ms 2025-03-28 22:26:16,157 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-33369321 with expectation True passed 2025-03-28 22:26:16,220 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:26:16,220 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-03-28 22:26:16,220 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:26:16,221 - INFO - wget check of of http://10.47.255.251 from ctest-pod-37649715 passed 2025-03-28 22:26:16,346 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:26:16,346 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-03-28 22:26:16,346 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:26:16,346 - INFO - wget check of of http://10.47.255.250 from ctest-pod-37649715 passed 2025-03-28 22:26:21,413 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-03-28 22:26:21,414 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-03-28 22:26:21,414 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-03-28 22:26:21,414 - INFO - wget check of of http://10.47.255.250 from ctest-pod-23078103 passed 2025-03-28 22:26:26,487 - DEBUG - [Pod ctest-pod-49877151] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-03-28 22:26:26,487 - DEBUG - [Pod Pod ctest-pod-49877151] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-03-28 22:26:26,487 - DEBUG - [Pod Pod ctest-pod-49877151] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-03-28 22:26:26,487 - INFO - wget check of of http://10.47.255.251 from ctest-pod-49877151 passed 2025-03-28 22:26:26,494 - INFO - Updating Network Policy ingress-on-service 2025-03-28 22:26:31,570 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:31,570 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:31,570 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:36,635 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:36,635 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:36,635 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:41,696 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:41,696 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:41,696 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:46,772 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:46,772 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:46,772 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:51,832 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:51,832 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:51,832 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:56,880 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:56,881 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:26:56,881 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:01,958 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:01,958 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:01,958 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:07,024 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:07,024 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:07,024 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:12,085 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:12,086 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:12,086 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:17,155 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:17,155 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:17,155 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) wget: download timed out 2025-03-28 22:27:17,155 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-62058388': 0, 'ctest-pod-86807875': 0} 2025-03-28 22:27:17,232 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,233 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:27:17,233 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,293 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: ctest-pod-62058388 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,293 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:27:17,293 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: ctest-pod-62058388 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,293 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-62058388': 1, 'ctest-pod-86807875': 1} 2025-03-28 22:27:17,369 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,369 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:27:17,369 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: ctest-pod-86807875 Connecting to 10.107.229.103:80 (10.107.229.103:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,437 - DEBUG - [Pod ctest-pod-33369321] Cmd: wget http://10.107.229.103:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.229.103:80 (10.107.229.103:80) ctest-pod-62058388 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,437 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd wget http://10.107.229.103:80/ -O - -T 5 -t 1 passed 2025-03-28 22:27:17,437 - DEBUG - [Pod Pod ctest-pod-33369321] Cmd output: Connecting to 10.107.229.103:80 (10.107.229.103:80) ctest-pod-62058388 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-03-28 22:27:17,437 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-62058388': 1, 'ctest-pod-86807875': 1} 2025-03-28 22:27:19,513 - DEBUG - [Pod ctest-pod-37649715] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.938 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.085 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.088 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.085/0.370/0.938 ms 2025-03-28 22:27:19,513 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-37649715 with expectation True passed 2025-03-28 22:27:21,573 - DEBUG - [Pod ctest-pod-23078103] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.741 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.076 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.091 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.076/0.302/0.741 ms 2025-03-28 22:27:21,573 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-23078103 with expectation True passed 2025-03-28 22:27:23,644 - DEBUG - [Pod ctest-pod-33369321] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.139 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.100 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.190 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.100/0.476/1.139 ms 2025-03-28 22:27:23,644 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-33369321 with expectation True passed 2025-03-28 22:27:23,708 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:27:23,708 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-03-28 22:27:23,708 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:27:23,708 - INFO - wget check of of http://10.47.255.251 from ctest-pod-37649715 passed 2025-03-28 22:27:23,768 - DEBUG - [Pod ctest-pod-37649715] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:27:23,768 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-03-28 22:27:23,768 - DEBUG - [Pod Pod ctest-pod-37649715] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:27:23,768 - INFO - wget check of of http://10.47.255.250 from ctest-pod-37649715 passed 2025-03-28 22:27:28,842 - DEBUG - [Pod ctest-pod-23078103] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-03-28 22:27:28,842 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-03-28 22:27:28,842 - DEBUG - [Pod Pod ctest-pod-23078103] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-03-28 22:27:28,842 - INFO - wget check of of http://10.47.255.250 from ctest-pod-23078103 passed 2025-03-28 22:27:28,905 - DEBUG - [Pod ctest-pod-49877151] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:27:28,905 - DEBUG - [Pod Pod ctest-pod-49877151] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-03-28 22:27:28,905 - DEBUG - [Pod Pod ctest-pod-49877151] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-03-28 22:27:28,905 - INFO - wget check of of http://10.47.255.251 from ctest-pod-49877151 passed 2025-03-28 22:27:28,906 - INFO - Deleting Network Policy : ingress-on-service 2025-03-28 22:27:28,906 - INFO - Deleting service : ctest-nginx-svc-94824018 2025-03-28 22:27:29,148 - DEBUG - No XMPP flaps were noticed during the test 2025-03-28 22:27:29,148 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:15] 2025-03-28 22:27:29,148 - INFO - -------------------------------------------------------------------------------- 2025-03-28 22:27:29,152 - INFO - Deleting pod ctest-new-default-20637346:ctest-pod-62058388 2025-03-28 22:27:29,153 - INFO - Deleting pod ctest-new-default-20637346:ctest-pod-86807875 2025-03-28 22:27:29,153 - INFO - Deleting pod ctest-non-default-05997853:ctest-pod-72014971 2025-03-28 22:27:29,153 - INFO - Deleting pod ctest-non-default-05997853:ctest-pod-41237285 2025-03-28 22:27:29,154 - INFO - Deleting pod ctest-new-default-20637346:ctest-pod-23078103 2025-03-28 22:27:29,154 - INFO - Deleting pod ctest-new-default-20637346:ctest-pod-37649715 2025-03-28 22:27:29,154 - INFO - Deleting pod ctest-non-default-05997853:ctest-pod-33369321 2025-03-28 22:27:29,155 - INFO - Deleting pod ctest-non-default-05997853:ctest-pod-49877151 2025-03-28 22:27:29,272 - WARNING - Pod uuid 64b25dab-872b-4dab-9b60-554bcb1e7402 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,346 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,349 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,349 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,350 - WARNING - Pod uuid 091059e8-53d3-4a47-b2b7-7d1c27276a92 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,350 - WARNING - Pod uuid 7ba28042-ca7d-4833-ae19-b7891762cd22 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,351 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:29,351 - WARNING - Pod uuid bd417630-b197-4e07-9e35-7a8127a2610f is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:34,281 - DEBUG - Pod 64b25dab-872b-4dab-9b60-554bcb1e7402 is not in agent 10.0.0.24 VM list 2025-03-28 22:27:34,287 - DEBUG - VMI 55b6e4ca-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:27:34,287 - INFO - Verified that pod ctest-pod-62058388 is removed in agent 2025-03-28 22:27:34,361 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:34,378 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:34,380 - DEBUG - Pod 7ba28042-ca7d-4833-ae19-b7891762cd22 is not in agent 10.0.0.24 VM list 2025-03-28 22:27:34,381 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:34,381 - DEBUG - Pod 091059e8-53d3-4a47-b2b7-7d1c27276a92 is not in agent 10.0.0.24 VM list 2025-03-28 22:27:34,383 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:34,383 - DEBUG - Pod bd417630-b197-4e07-9e35-7a8127a2610f is not in agent 10.0.0.24 VM list 2025-03-28 22:27:34,394 - DEBUG - VMI 56e35720-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:27:34,394 - INFO - Verified that pod ctest-pod-72014971 is removed in agent 2025-03-28 22:27:34,395 - DEBUG - VMI 563a600c-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:27:34,395 - INFO - Verified that pod ctest-pod-86807875 is removed in agent 2025-03-28 22:27:34,395 - DEBUG - VMI 57650b4e-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:27:34,396 - INFO - Verified that pod ctest-pod-41237285 is removed in agent 2025-03-28 22:27:39,370 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:39,386 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:39,395 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:39,396 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:44,387 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:44,394 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:44,408 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:44,408 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:49,404 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:49,405 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:49,420 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:49,420 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:54,417 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:54,418 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:54,430 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:54,431 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:59,490 - WARNING - Pod uuid 9647793b-be0e-48e6-ac5d-418bd8b17050 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:59,491 - WARNING - Pod uuid 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:59,492 - WARNING - Pod uuid f7a0187a-2ad5-4b38-8ba2-2685740e1510 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:27:59,492 - WARNING - Pod uuid 59972a4f-446c-4b82-bdf2-f379df51b269 is still seen in agent 10.0.0.24 VM list 2025-03-28 22:28:04,510 - DEBUG - Pod 9647793b-be0e-48e6-ac5d-418bd8b17050 is not in agent 10.0.0.24 VM list 2025-03-28 22:28:04,513 - DEBUG - Pod 59972a4f-446c-4b82-bdf2-f379df51b269 is not in agent 10.0.0.24 VM list 2025-03-28 22:28:04,515 - DEBUG - Pod 7632dfa5-7f71-4193-a4ce-48105d9f9d36 is not in agent 10.0.0.24 VM list 2025-03-28 22:28:04,516 - DEBUG - Pod f7a0187a-2ad5-4b38-8ba2-2685740e1510 is not in agent 10.0.0.24 VM list 2025-03-28 22:28:04,531 - DEBUG - VMI 5863d0d4-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:28:04,531 - INFO - Verified that pod ctest-pod-37649715 is removed in agent 2025-03-28 22:28:04,533 - DEBUG - VMI 595913aa-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:28:04,533 - INFO - Verified that pod ctest-pod-49877151 is removed in agent 2025-03-28 22:28:04,533 - DEBUG - VMI 58dd0404-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:28:04,533 - INFO - Verified that pod ctest-pod-33369321 is removed in agent 2025-03-28 22:28:04,533 - DEBUG - VMI 57ed7a1a-0c23-11f0-ac62-fa163e221e23 is removed from agent 10.0.0.24 2025-03-28 22:28:04,533 - INFO - Verified that pod ctest-pod-23078103 is removed in agent 2025-03-28 22:28:04,534 - INFO - Deleting namespace ctest-non-default-05997853 2025-03-28 22:28:04,559 - DEBUG - Namespace ctest-non-default-05997853 still in k8s 2025-03-28 22:28:06,563 - DEBUG - Namespace ctest-non-default-05997853 still in k8s 2025-03-28 22:28:08,568 - DEBUG - Namespace ctest-non-default-05997853 still in k8s 2025-03-28 22:28:10,576 - DEBUG - Namespace ctest-non-default-05997853 is not in k8s 2025-03-28 22:28:10,576 - DEBUG - No need to check project deletion in contrail-api 2025-03-28 22:28:10,576 - INFO - Deleting namespace ctest-new-default-20637346 2025-03-28 22:28:10,594 - DEBUG - Namespace ctest-new-default-20637346 still in k8s 2025-03-28 22:28:12,600 - DEBUG - Namespace ctest-new-default-20637346 still in k8s 2025-03-28 22:28:14,606 - DEBUG - Namespace ctest-new-default-20637346 still in k8s 2025-03-28 22:28:16,610 - DEBUG - Namespace ctest-new-default-20637346 is not in k8s 2025-03-28 22:28:16,610 - DEBUG - No need to check project deletion in contrail-api