2024-08-25 13:56:29,469 - INFO - Using existing project ['default-domain', 'k8s-default'](8681f186-e3f5-4784-8e20-c9cc79ba6916) 2024-08-25 13:56:29,500 - DEBUG - Namespace ctest-new-default-00658966 not present 2024-08-25 13:56:29,513 - INFO - Created namespace ctest-new-default-00658966 2024-08-25 13:56:32,524 - DEBUG - Namespace ctest-non-default-01186164 not present 2024-08-25 13:56:32,550 - INFO - Created namespace ctest-non-default-01186164 2024-08-25 13:56:35,608 - DEBUG - Pod ctest-pod-49439366 not present 2024-08-25 13:56:35,609 - INFO - Creating Pod ctest-pod-49439366 2024-08-25 13:56:35,668 - DEBUG - Pod : ctest-pod-49439366 UUID is 4dacf449-ae49-47a4-a0a5-747a2dec7e8d 2024-08-25 13:56:35,676 - DEBUG - Pod ctest-pod-14278928 not present 2024-08-25 13:56:35,676 - INFO - Creating Pod ctest-pod-14278928 2024-08-25 13:56:35,707 - DEBUG - Pod : ctest-pod-14278928 UUID is e3a4c5a8-a521-44de-8452-01202776b86b 2024-08-25 13:56:35,711 - DEBUG - Pod ctest-pod-84088809 not present 2024-08-25 13:56:35,711 - INFO - Creating Pod ctest-pod-84088809 2024-08-25 13:56:35,745 - DEBUG - Pod : ctest-pod-84088809 UUID is a385f94e-ee9f-4fbb-9ec7-0128780e02e4 2024-08-25 13:56:35,749 - DEBUG - Pod ctest-pod-52961615 not present 2024-08-25 13:56:35,749 - INFO - Creating Pod ctest-pod-52961615 2024-08-25 13:56:35,780 - DEBUG - Pod : ctest-pod-52961615 UUID is 2c0da65c-ee93-4ef3-bd64-a61792dbcdf6 2024-08-25 13:56:35,783 - DEBUG - Pod ctest-pod-92321120 not present 2024-08-25 13:56:35,784 - INFO - Creating Pod ctest-pod-92321120 2024-08-25 13:56:35,811 - DEBUG - Pod : ctest-pod-92321120 UUID is 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a 2024-08-25 13:56:35,813 - DEBUG - Pod ctest-pod-66797961 not present 2024-08-25 13:56:35,813 - INFO - Creating Pod ctest-pod-66797961 2024-08-25 13:56:35,839 - DEBUG - Pod : ctest-pod-66797961 UUID is 6ffcb705-b443-46c9-b83a-daa36792d20b 2024-08-25 13:56:35,844 - DEBUG - Pod ctest-pod-49731768 not present 2024-08-25 13:56:35,844 - INFO - Creating Pod ctest-pod-49731768 2024-08-25 13:56:35,864 - DEBUG - Pod : ctest-pod-49731768 UUID is 69a07f84-407d-43f8-8b09-9e52a7be91d2 2024-08-25 13:56:35,867 - DEBUG - Pod ctest-pod-16040972 not present 2024-08-25 13:56:35,867 - INFO - Creating Pod ctest-pod-16040972 2024-08-25 13:56:35,887 - DEBUG - Pod : ctest-pod-16040972 UUID is 50233047-e78e-4449-844a-c8a66b7bc327 2024-08-25 13:56:36,126 - INFO - Project default-domain:k8s-ctest-new-default-00658966 for Namespace ctest-new-default-00658966 is seen in contrail-api 2024-08-25 13:56:36,127 - DEBUG - Project uuid in Contrail is e33a97e6-bdc1-4163-b1dc-e2e99c073673 2024-08-25 13:56:36,382 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_NamespaceDatabaseList?namespace_uuid=7b2318d5-f13a-463e-8b7d-190061cbda3d 2024-08-25 13:56:36,388 - INFO - Namespace ctest-new-default-00658966 with uuid 7b2318d5-f13a-463e-8b7d-190061cbda3d found in kube manager 2024-08-25 13:56:36,388 - INFO - Namespace ctest-new-default-00658966 verification passed 2024-08-25 13:56:36,675 - INFO - Project default-domain:k8s-ctest-non-default-01186164 for Namespace ctest-non-default-01186164 is seen in contrail-api 2024-08-25 13:56:36,676 - DEBUG - Project uuid in Contrail is 40d4e0bd-645e-43ef-9409-1384ab503e65 2024-08-25 13:56:36,676 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_NamespaceDatabaseList?namespace_uuid=73097353-f63d-498d-93ba-90a508a8035d 2024-08-25 13:56:36,680 - INFO - Namespace ctest-non-default-01186164 with uuid 73097353-f63d-498d-93ba-90a508a8035d found in kube manager 2024-08-25 13:56:36,681 - INFO - Namespace ctest-non-default-01186164 verification passed 2024-08-25 13:56:36,687 - DEBUG - Pod ctest-pod-49439366 not in running state.Currently in Pending 2024-08-25 13:56:41,696 - DEBUG - Pod ctest-pod-49439366 not in running state.Currently in Pending 2024-08-25 13:56:46,709 - INFO - Pod ctest-pod-49439366 is in running state.Got IP 10.47.255.248 2024-08-25 13:56:46,777 - DEBUG - Pod ctest-pod-49439366 has vmi d97aa32a-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:56:46,777 - INFO - Verified pod ctest-pod-49439366 in contrail-api 2024-08-25 13:56:46,812 - DEBUG - VMI d97aa32a-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:56:46,813 - INFO - Verified Pod ctest-pod-49439366 in agent 10.0.0.12 2024-08-25 13:56:46,813 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=4dacf449-ae49-47a4-a0a5-747a2dec7e8d 2024-08-25 13:56:46,819 - INFO - Pod ctest-pod-49439366 with uuid 4dacf449-ae49-47a4-a0a5-747a2dec7e8d found in kube manager 2024-08-25 13:56:46,819 - INFO - Pod ctest-pod-49439366 verification passed 2024-08-25 13:56:46,826 - DEBUG - Pod ctest-pod-14278928 not in running state.Currently in Pending 2024-08-25 13:56:51,837 - INFO - Pod ctest-pod-14278928 is in running state.Got IP 10.47.255.247 2024-08-25 13:56:51,908 - DEBUG - Pod ctest-pod-14278928 has vmi da2d0e3e-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:56:51,908 - INFO - Verified pod ctest-pod-14278928 in contrail-api 2024-08-25 13:56:51,929 - DEBUG - VMI da2d0e3e-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:56:51,929 - INFO - Verified Pod ctest-pod-14278928 in agent 10.0.0.12 2024-08-25 13:56:51,930 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=e3a4c5a8-a521-44de-8452-01202776b86b 2024-08-25 13:56:51,935 - INFO - Pod ctest-pod-14278928 with uuid e3a4c5a8-a521-44de-8452-01202776b86b found in kube manager 2024-08-25 13:56:51,935 - INFO - Pod ctest-pod-14278928 verification passed 2024-08-25 13:56:51,948 - DEBUG - Pod ctest-pod-84088809 not in running state.Currently in Pending 2024-08-25 13:56:56,968 - INFO - Pod ctest-pod-84088809 is in running state.Got IP 10.47.255.246 2024-08-25 13:56:57,037 - DEBUG - Pod ctest-pod-84088809 has vmi db5d959e-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:56:57,037 - INFO - Verified pod ctest-pod-84088809 in contrail-api 2024-08-25 13:56:57,072 - DEBUG - VMI db5d959e-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:56:57,073 - INFO - Verified Pod ctest-pod-84088809 in agent 10.0.0.12 2024-08-25 13:56:57,074 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=a385f94e-ee9f-4fbb-9ec7-0128780e02e4 2024-08-25 13:56:57,088 - INFO - Pod ctest-pod-84088809 with uuid a385f94e-ee9f-4fbb-9ec7-0128780e02e4 found in kube manager 2024-08-25 13:56:57,089 - INFO - Pod ctest-pod-84088809 verification passed 2024-08-25 13:56:57,108 - INFO - Pod ctest-pod-52961615 is in running state.Got IP 10.47.255.245 2024-08-25 13:56:57,182 - DEBUG - Pod ctest-pod-52961615 has vmi dc1d4b14-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:56:57,183 - INFO - Verified pod ctest-pod-52961615 in contrail-api 2024-08-25 13:56:57,224 - DEBUG - VMI dc1d4b14-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:56:57,225 - INFO - Verified Pod ctest-pod-52961615 in agent 10.0.0.12 2024-08-25 13:56:57,225 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=2c0da65c-ee93-4ef3-bd64-a61792dbcdf6 2024-08-25 13:56:57,238 - INFO - Pod ctest-pod-52961615 with uuid 2c0da65c-ee93-4ef3-bd64-a61792dbcdf6 found in kube manager 2024-08-25 13:56:57,239 - INFO - Pod ctest-pod-52961615 verification passed 2024-08-25 13:56:57,259 - DEBUG - Pod ctest-pod-92321120 not in running state.Currently in Pending 2024-08-25 13:57:02,283 - INFO - Pod ctest-pod-92321120 is in running state.Got IP 10.47.255.244 2024-08-25 13:57:02,356 - DEBUG - Pod ctest-pod-92321120 has vmi dcfadd4e-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:57:02,357 - INFO - Verified pod ctest-pod-92321120 in contrail-api 2024-08-25 13:57:02,401 - DEBUG - VMI dcfadd4e-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:57:02,402 - INFO - Verified Pod ctest-pod-92321120 in agent 10.0.0.12 2024-08-25 13:57:02,403 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a 2024-08-25 13:57:02,417 - INFO - Pod ctest-pod-92321120 with uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a found in kube manager 2024-08-25 13:57:02,417 - INFO - Pod ctest-pod-92321120 verification passed 2024-08-25 13:57:02,439 - INFO - Pod ctest-pod-49731768 is in running state.Got IP 10.47.255.242 2024-08-25 13:57:02,511 - DEBUG - Pod ctest-pod-49731768 has vmi de83e520-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:57:02,511 - INFO - Verified pod ctest-pod-49731768 in contrail-api 2024-08-25 13:57:02,548 - DEBUG - VMI de83e520-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:57:02,549 - INFO - Verified Pod ctest-pod-49731768 in agent 10.0.0.12 2024-08-25 13:57:02,549 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=69a07f84-407d-43f8-8b09-9e52a7be91d2 2024-08-25 13:57:02,562 - INFO - Pod ctest-pod-49731768 with uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 found in kube manager 2024-08-25 13:57:02,562 - INFO - Pod ctest-pod-49731768 verification passed 2024-08-25 13:57:02,581 - INFO - Pod ctest-pod-66797961 is in running state.Got IP 10.47.255.243 2024-08-25 13:57:02,652 - DEBUG - Pod ctest-pod-66797961 has vmi ddbe9324-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:57:02,653 - INFO - Verified pod ctest-pod-66797961 in contrail-api 2024-08-25 13:57:02,687 - DEBUG - VMI ddbe9324-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:57:02,688 - INFO - Verified Pod ctest-pod-66797961 in agent 10.0.0.12 2024-08-25 13:57:02,688 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=6ffcb705-b443-46c9-b83a-daa36792d20b 2024-08-25 13:57:02,699 - INFO - Pod ctest-pod-66797961 with uuid 6ffcb705-b443-46c9-b83a-daa36792d20b found in kube manager 2024-08-25 13:57:02,699 - INFO - Pod ctest-pod-66797961 verification passed 2024-08-25 13:57:02,718 - INFO - Pod ctest-pod-16040972 is in running state.Got IP 10.47.255.241 2024-08-25 13:57:02,777 - DEBUG - Pod ctest-pod-16040972 has vmi df433aa6-62e9-11ef-8e3c-fa163edc4911 2024-08-25 13:57:02,777 - INFO - Verified pod ctest-pod-16040972 in contrail-api 2024-08-25 13:57:02,810 - DEBUG - VMI df433aa6-62e9-11ef-8e3c-fa163edc4911 is active in agent 10.0.0.12 2024-08-25 13:57:02,811 - INFO - Verified Pod ctest-pod-16040972 in agent 10.0.0.12 2024-08-25 13:57:02,811 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_VirtualMachineDatabaseList?x=50233047-e78e-4449-844a-c8a66b7bc327 2024-08-25 13:57:02,820 - INFO - Pod ctest-pod-16040972 with uuid 50233047-e78e-4449-844a-c8a66b7bc327 found in kube manager 2024-08-25 13:57:02,820 - INFO - Pod ctest-pod-16040972 verification passed 2024-08-25 13:57:02,821 - INFO - ================================================================================ 2024-08-25 13:57:02,821 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-08-25 13:57:02,821 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-08-25 13:57:03,364 - DEBUG - Nothing to compare xmpp stats {'10.0.0.12': {'10.0.0.12': '0'}} with 2024-08-25 13:57:03,364 - INFO - Initial checks done. Running the testcase now 2024-08-25 13:57:03,364 - INFO - 2024-08-25 13:57:03,447 - DEBUG - [Pod ctest-pod-49439366] Cmd: echo ctest-pod-49439366 > /usr/share/nginx/html/index.html, Output: 2024-08-25 13:57:03,499 - DEBUG - [Pod ctest-pod-14278928] Cmd: echo ctest-pod-14278928 > /usr/share/nginx/html/index.html, Output: 2024-08-25 13:57:03,551 - DEBUG - [Pod ctest-pod-84088809] Cmd: echo ctest-pod-84088809 > /usr/share/nginx/html/index.html, Output: 2024-08-25 13:57:03,597 - DEBUG - [Pod ctest-pod-52961615] Cmd: echo ctest-pod-52961615 > /usr/share/nginx/html/index.html, Output: 2024-08-25 13:57:03,601 - DEBUG - Service ctest-nginx-svc-35406498 not present 2024-08-25 13:57:03,601 - INFO - Creating service ctest-nginx-svc-35406498 2024-08-25 13:57:03,616 - INFO - Created Service ctest-nginx-svc-35406498 2024-08-25 13:57:03,631 - WARNING - Service UUID 509a4e2f-b358-4a63-a96e-f666a600fba1 not yet found in contrail-api 2024-08-25 13:57:04,646 - INFO - Validated that Service ctest-nginx-svc-35406498 is seen in contrail-api 2024-08-25 13:57:04,646 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_LoadbalancerDatabaseList?x=509a4e2f-b358-4a63-a96e-f666a600fba1 2024-08-25 13:57:04,655 - INFO - Service ctest-nginx-svc-35406498 with uuid 509a4e2f-b358-4a63-a96e-f666a600fba1 found in kube manager 2024-08-25 13:57:04,656 - INFO - Service ctest-nginx-svc-35406498 verification passed 2024-08-25 13:57:05,780 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 13:57:05,780 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:05,780 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 13:57:05,855 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:05,855 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:05,855 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:05,945 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: ctest-pod-14278928 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:05,945 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:05,945 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: ctest-pod-14278928 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:05,995 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:05,995 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:05,995 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:06,058 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-49439366 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:06,058 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:06,058 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-49439366 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:06,058 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-49439366': 1, 'ctest-pod-14278928': 4} 2024-08-25 13:57:06,130 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 13:57:06,131 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:06,131 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 13:57:06,179 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 13:57:06,180 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:06,180 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 13:57:06,231 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: ctest-pod-49439366 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:06,231 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:57:06,231 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: ctest-pod-49439366 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:57:06,231 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-49439366': 1, 'ctest-pod-14278928': 2} 2024-08-25 13:57:06,235 - DEBUG - Network policy ingress-on-service not present 2024-08-25 13:57:06,236 - INFO - Creating Network Policy ingress-on-service 2024-08-25 13:57:06,246 - INFO - Network policy found in k8s 2024-08-25 13:57:06,247 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=add5c97b-ab68-429d-ad54-cadddd08c2de 2024-08-25 13:57:06,254 - INFO - Network Policy ingress-on-service with uuid add5c97b-ab68-429d-ad54-cadddd08c2de found in kube manager 2024-08-25 13:57:06,283 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-08-25 13:57:06,283 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-08-25 13:57:06,283 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=add5c97b-ab68-429d-ad54-cadddd08c2de 2024-08-25 13:57:06,325 - WARNING - Network policy with name ingress-on-service not found in agent 2024-08-25 13:57:08,326 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=add5c97b-ab68-429d-ad54-cadddd08c2de 2024-08-25 13:57:08,371 - WARNING - Network policy with name ingress-on-service not found in agent 2024-08-25 13:57:10,373 - DEBUG - Requesting: http://10.0.0.12:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=add5c97b-ab68-429d-ad54-cadddd08c2de 2024-08-25 13:57:10,460 - INFO - Network policy with name ingress-on-service found in agent 2024-08-25 13:57:10,461 - INFO - Network Policy ingress-on-service verification passed 2024-08-25 13:57:15,553 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:15,553 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:15,553 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:20,662 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:20,662 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:20,662 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:25,758 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:25,758 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:25,758 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:30,838 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:30,838 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:30,839 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:35,950 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:35,950 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:35,951 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:41,021 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:41,022 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:41,022 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:46,117 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:46,118 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:46,118 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:51,230 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:51,230 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:51,230 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:56,349 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:56,349 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:57:56,349 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:01,420 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:01,421 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:01,421 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:01,422 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-49439366': 0, 'ctest-pod-14278928': 0} 2024-08-25 13:58:01,558 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49439366 2024-08-25 13:58:01,559 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:58:01,559 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49439366 2024-08-25 13:58:01,649 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49439366 2024-08-25 13:58:01,649 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:58:01,649 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49439366 2024-08-25 13:58:01,721 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49439366 2024-08-25 13:58:01,721 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:58:01,721 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49439366 2024-08-25 13:58:01,783 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:58:01,783 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 13:58:01,783 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 13:58:01,783 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-49439366': 3, 'ctest-pod-14278928': 1} 2024-08-25 13:58:06,848 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:06,848 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:06,848 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:11,999 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:12,000 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:12,000 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:17,128 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:17,128 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:17,128 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:22,258 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:22,258 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:22,258 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:27,379 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:27,380 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:27,380 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:32,489 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:32,489 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:32,489 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:37,620 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:37,621 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:37,621 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:42,746 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:42,747 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:42,747 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:47,877 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:47,878 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:47,878 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:53,013 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:53,013 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:53,014 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:58:53,014 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-49439366': 0, 'ctest-pod-14278928': 0} 2024-08-25 13:58:55,100 - DEBUG - [Pod ctest-pod-66797961] Cmd: ping -c 3 10.47.255.244, Output: PING 10.47.255.244 (10.47.255.244): 56 data bytes 64 bytes from 10.47.255.244: seq=0 ttl=63 time=0.861 ms 64 bytes from 10.47.255.244: seq=1 ttl=63 time=0.063 ms 64 bytes from 10.47.255.244: seq=2 ttl=63 time=0.062 ms --- 10.47.255.244 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.062/0.328/0.861 ms 2024-08-25 13:58:55,100 - INFO - Ping check to IP 10.47.255.244 from pod ctest-pod-66797961 with expectation True passed 2024-08-25 13:58:57,162 - DEBUG - [Pod ctest-pod-92321120] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=1.314 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.103 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.120 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.103/0.512/1.314 ms 2024-08-25 13:58:57,163 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-92321120 with expectation True passed 2024-08-25 13:58:59,291 - DEBUG - [Pod ctest-pod-49731768] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=0.667 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.150 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.250 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.150/0.355/0.667 ms 2024-08-25 13:58:59,291 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-49731768 with expectation True passed 2024-08-25 13:58:59,409 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 13:58:59,409 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 passed 2024-08-25 13:58:59,409 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 13:58:59,409 - INFO - wget check of of http://10.47.255.248 from ctest-pod-66797961 passed 2024-08-25 13:58:59,482 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 13:58:59,482 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 passed 2024-08-25 13:58:59,482 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 13:58:59,483 - INFO - wget check of of http://10.47.255.247 from ctest-pod-66797961 passed 2024-08-25 13:59:04,542 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2024-08-25 13:59:04,543 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2024-08-25 13:59:04,543 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2024-08-25 13:59:04,543 - INFO - wget check of of http://10.47.255.247 from ctest-pod-92321120 passed 2024-08-25 13:59:09,662 - DEBUG - [Pod ctest-pod-16040972] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) wget: download timed out 2024-08-25 13:59:09,662 - DEBUG - [Pod Pod ctest-pod-16040972] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.248 (10.47.255.248:80) wget: download timed out 2024-08-25 13:59:09,662 - DEBUG - [Pod Pod ctest-pod-16040972] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) wget: download timed out 2024-08-25 13:59:09,662 - INFO - wget check of of http://10.47.255.248 from ctest-pod-16040972 passed 2024-08-25 13:59:09,687 - INFO - Updating Network Policy ingress-on-service 2024-08-25 13:59:14,801 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:14,801 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:14,801 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:19,889 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:19,890 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:19,890 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:25,021 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:25,021 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:25,021 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:30,143 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:30,144 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:30,144 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:35,208 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:35,208 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:35,209 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:40,334 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:40,334 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:40,334 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:45,417 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:45,417 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:45,417 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:50,472 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:50,472 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:50,472 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:55,593 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:55,593 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 13:59:55,594 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 14:00:00,655 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 14:00:00,655 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 14:00:00,655 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) wget: download timed out 2024-08-25 14:00:00,656 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-49439366': 0, 'ctest-pod-14278928': 0} 2024-08-25 14:00:00,716 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,716 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 14:00:00,716 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) ctest-pod-14278928 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,770 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: ctest-pod-49439366 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,770 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 14:00:00,770 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: ctest-pod-49439366 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,770 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-49439366': 1, 'ctest-pod-14278928': 1} 2024-08-25 14:00:00,816 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 14:00:00,816 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 14:00:00,816 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-14278928 2024-08-25 14:00:00,880 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: ctest-pod-14278928 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,880 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 14:00:00,880 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: ctest-pod-14278928 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,927 - DEBUG - [Pod ctest-pod-49731768] Cmd: wget http://10.96.109.84:80/ -O - -T 5 -t 1 , Output: ctest-pod-49439366 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,927 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd wget http://10.96.109.84:80/ -O - -T 5 -t 1 passed 2024-08-25 14:00:00,927 - DEBUG - [Pod Pod ctest-pod-49731768] Cmd output: ctest-pod-49439366 Connecting to 10.96.109.84:80 (10.96.109.84:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-25 14:00:00,927 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-49439366': 1, 'ctest-pod-14278928': 2} 2024-08-25 14:00:02,989 - DEBUG - [Pod ctest-pod-66797961] Cmd: ping -c 3 10.47.255.244, Output: PING 10.47.255.244 (10.47.255.244): 56 data bytes 64 bytes from 10.47.255.244: seq=0 ttl=63 time=0.626 ms 64 bytes from 10.47.255.244: seq=1 ttl=63 time=0.104 ms 64 bytes from 10.47.255.244: seq=2 ttl=63 time=0.192 ms --- 10.47.255.244 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.104/0.307/0.626 ms 2024-08-25 14:00:02,989 - INFO - Ping check to IP 10.47.255.244 from pod ctest-pod-66797961 with expectation True passed 2024-08-25 14:00:05,102 - DEBUG - [Pod ctest-pod-92321120] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=1.501 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.126 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.134 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.126/0.587/1.501 ms 2024-08-25 14:00:05,102 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-92321120 with expectation True passed 2024-08-25 14:00:07,210 - DEBUG - [Pod ctest-pod-49731768] Cmd: ping -c 3 10.47.255.243, Output: PING 10.47.255.243 (10.47.255.243): 56 data bytes 64 bytes from 10.47.255.243: seq=0 ttl=63 time=0.784 ms 64 bytes from 10.47.255.243: seq=1 ttl=63 time=0.104 ms 64 bytes from 10.47.255.243: seq=2 ttl=63 time=0.191 ms --- 10.47.255.243 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.104/0.359/0.784 ms 2024-08-25 14:00:07,210 - INFO - Ping check to IP 10.47.255.243 from pod ctest-pod-49731768 with expectation True passed 2024-08-25 14:00:07,318 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 14:00:07,318 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 passed 2024-08-25 14:00:07,318 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 14:00:07,319 - INFO - wget check of of http://10.47.255.248 from ctest-pod-66797961 passed 2024-08-25 14:00:07,385 - DEBUG - [Pod ctest-pod-66797961] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 14:00:07,385 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 passed 2024-08-25 14:00:07,385 - DEBUG - [Pod Pod ctest-pod-66797961] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 14:00:07,385 - INFO - wget check of of http://10.47.255.247 from ctest-pod-66797961 passed 2024-08-25 14:00:12,455 - DEBUG - [Pod ctest-pod-92321120] Cmd: wget http://10.47.255.247 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2024-08-25 14:00:12,456 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd wget http://10.47.255.247 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2024-08-25 14:00:12,456 - DEBUG - [Pod Pod ctest-pod-92321120] Cmd output: Connecting to 10.47.255.247 (10.47.255.247:80) wget: download timed out 2024-08-25 14:00:12,456 - INFO - wget check of of http://10.47.255.247 from ctest-pod-92321120 passed 2024-08-25 14:00:12,566 - DEBUG - [Pod ctest-pod-16040972] Cmd: wget http://10.47.255.248 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 14:00:12,567 - DEBUG - [Pod Pod ctest-pod-16040972] Cmd wget http://10.47.255.248 -O /dev/null -T 5 -t 1 passed 2024-08-25 14:00:12,567 - DEBUG - [Pod Pod ctest-pod-16040972] Cmd output: Connecting to 10.47.255.248 (10.47.255.248:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-25 14:00:12,567 - INFO - wget check of of http://10.47.255.248 from ctest-pod-16040972 passed 2024-08-25 14:00:12,567 - INFO - Deleting Network Policy : ingress-on-service 2024-08-25 14:00:12,569 - INFO - Deleting service : ctest-nginx-svc-35406498 2024-08-25 14:00:13,406 - DEBUG - No XMPP flaps were noticed during the test 2024-08-25 14:00:13,407 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:11] 2024-08-25 14:00:13,407 - INFO - -------------------------------------------------------------------------------- 2024-08-25 14:00:13,428 - INFO - Deleting pod ctest-new-default-00658966:ctest-pod-49439366 2024-08-25 14:00:13,429 - INFO - Deleting pod ctest-new-default-00658966:ctest-pod-14278928 2024-08-25 14:00:13,430 - INFO - Deleting pod ctest-non-default-01186164:ctest-pod-84088809 2024-08-25 14:00:13,432 - INFO - Deleting pod ctest-non-default-01186164:ctest-pod-52961615 2024-08-25 14:00:13,433 - INFO - Deleting pod ctest-new-default-00658966:ctest-pod-92321120 2024-08-25 14:00:13,434 - INFO - Deleting pod ctest-new-default-00658966:ctest-pod-66797961 2024-08-25 14:00:13,435 - INFO - Deleting pod ctest-non-default-01186164:ctest-pod-49731768 2024-08-25 14:00:13,436 - INFO - Deleting pod ctest-non-default-01186164:ctest-pod-16040972 2024-08-25 14:00:13,691 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,692 - WARNING - Pod uuid 4dacf449-ae49-47a4-a0a5-747a2dec7e8d is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,694 - WARNING - Pod uuid e3a4c5a8-a521-44de-8452-01202776b86b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,698 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,698 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,700 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,703 - WARNING - Pod uuid 2c0da65c-ee93-4ef3-bd64-a61792dbcdf6 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:13,704 - WARNING - Pod uuid a385f94e-ee9f-4fbb-9ec7-0128780e02e4 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:18,807 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:18,811 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:18,814 - DEBUG - Pod 4dacf449-ae49-47a4-a0a5-747a2dec7e8d is not in agent 10.0.0.12 VM list 2024-08-25 14:00:18,823 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:18,825 - DEBUG - Pod e3a4c5a8-a521-44de-8452-01202776b86b is not in agent 10.0.0.12 VM list 2024-08-25 14:00:18,833 - DEBUG - Pod a385f94e-ee9f-4fbb-9ec7-0128780e02e4 is not in agent 10.0.0.12 VM list 2024-08-25 14:00:18,845 - DEBUG - Pod 2c0da65c-ee93-4ef3-bd64-a61792dbcdf6 is not in agent 10.0.0.12 VM list 2024-08-25 14:00:18,851 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:18,879 - DEBUG - VMI d97aa32a-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:18,879 - INFO - Verified that pod ctest-pod-49439366 is removed in agent 2024-08-25 14:00:18,882 - DEBUG - VMI dc1d4b14-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:18,882 - INFO - Verified that pod ctest-pod-52961615 is removed in agent 2024-08-25 14:00:18,883 - DEBUG - VMI db5d959e-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:18,883 - INFO - Verified that pod ctest-pod-84088809 is removed in agent 2024-08-25 14:00:18,884 - DEBUG - VMI da2d0e3e-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:18,884 - INFO - Verified that pod ctest-pod-14278928 is removed in agent 2024-08-25 14:00:23,869 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:23,870 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:23,871 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:23,886 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:28,945 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:28,950 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:28,951 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:28,953 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:33,997 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:34,004 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:34,005 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:34,006 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:39,090 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:39,093 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:39,101 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:39,103 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:44,111 - WARNING - Pod uuid 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:44,112 - WARNING - Pod uuid 69a07f84-407d-43f8-8b09-9e52a7be91d2 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:44,120 - WARNING - Pod uuid 6ffcb705-b443-46c9-b83a-daa36792d20b is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:44,121 - WARNING - Pod uuid 50233047-e78e-4449-844a-c8a66b7bc327 is still seen in agent 10.0.0.12 VM list 2024-08-25 14:00:49,179 - DEBUG - Pod 28f48d6f-84b3-4a38-ac89-e7f8fd5bc88a is not in agent 10.0.0.12 VM list 2024-08-25 14:00:49,196 - DEBUG - Pod 6ffcb705-b443-46c9-b83a-daa36792d20b is not in agent 10.0.0.12 VM list 2024-08-25 14:00:49,203 - DEBUG - Pod 50233047-e78e-4449-844a-c8a66b7bc327 is not in agent 10.0.0.12 VM list 2024-08-25 14:00:49,209 - DEBUG - Pod 69a07f84-407d-43f8-8b09-9e52a7be91d2 is not in agent 10.0.0.12 VM list 2024-08-25 14:00:49,260 - DEBUG - VMI df433aa6-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:49,260 - INFO - Verified that pod ctest-pod-16040972 is removed in agent 2024-08-25 14:00:49,262 - DEBUG - VMI ddbe9324-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:49,262 - INFO - Verified that pod ctest-pod-66797961 is removed in agent 2024-08-25 14:00:49,263 - DEBUG - VMI dcfadd4e-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:49,263 - INFO - Verified that pod ctest-pod-92321120 is removed in agent 2024-08-25 14:00:49,265 - DEBUG - VMI de83e520-62e9-11ef-8e3c-fa163edc4911 is removed from agent 10.0.0.12 2024-08-25 14:00:49,265 - INFO - Verified that pod ctest-pod-49731768 is removed in agent 2024-08-25 14:00:49,265 - INFO - Deleting namespace ctest-non-default-01186164 2024-08-25 14:00:49,286 - DEBUG - Namespace ctest-non-default-01186164 still in k8s 2024-08-25 14:00:51,298 - DEBUG - Namespace ctest-non-default-01186164 still in k8s 2024-08-25 14:00:53,303 - DEBUG - Namespace ctest-non-default-01186164 still in k8s 2024-08-25 14:00:55,310 - DEBUG - Namespace ctest-non-default-01186164 is not in k8s 2024-08-25 14:00:55,310 - DEBUG - No need to check project deletion in contrail-api 2024-08-25 14:00:55,310 - INFO - Deleting namespace ctest-new-default-00658966 2024-08-25 14:00:55,328 - DEBUG - Namespace ctest-new-default-00658966 still in k8s 2024-08-25 14:00:57,339 - DEBUG - Namespace ctest-new-default-00658966 still in k8s 2024-08-25 14:00:59,353 - DEBUG - Namespace ctest-new-default-00658966 still in k8s 2024-08-25 14:01:01,358 - DEBUG - Namespace ctest-new-default-00658966 is not in k8s 2024-08-25 14:01:01,358 - DEBUG - No need to check project deletion in contrail-api