2024-11-15 01:22:25,061 - INFO - Using existing project ['default-domain', 'k8s-default'](16a0499e-c99d-42f4-9e47-811296949bad) 2024-11-15 01:22:25,083 - DEBUG - Namespace ctest-new-default-76912177 not present 2024-11-15 01:22:25,122 - INFO - Created namespace ctest-new-default-76912177 2024-11-15 01:22:28,126 - DEBUG - Namespace ctest-non-default-80258725 not present 2024-11-15 01:22:28,137 - INFO - Created namespace ctest-non-default-80258725 2024-11-15 01:22:31,182 - DEBUG - Pod ctest-pod-06545866 not present 2024-11-15 01:22:31,182 - INFO - Creating Pod ctest-pod-06545866 2024-11-15 01:22:31,225 - DEBUG - Pod : ctest-pod-06545866 UUID is 5af10c86-9a51-4c0b-b120-893302f6dca8 2024-11-15 01:22:31,228 - DEBUG - Pod ctest-pod-67580768 not present 2024-11-15 01:22:31,228 - INFO - Creating Pod ctest-pod-67580768 2024-11-15 01:22:31,251 - DEBUG - Pod : ctest-pod-67580768 UUID is 633375d7-57c6-4dc5-8275-3be3cd82ea7d 2024-11-15 01:22:31,256 - DEBUG - Pod ctest-pod-64402717 not present 2024-11-15 01:22:31,256 - INFO - Creating Pod ctest-pod-64402717 2024-11-15 01:22:31,282 - DEBUG - Pod : ctest-pod-64402717 UUID is 36d3a14b-ad9d-4c97-9d04-1b2df6ceae6e 2024-11-15 01:22:31,286 - DEBUG - Pod ctest-pod-08133948 not present 2024-11-15 01:22:31,293 - INFO - Creating Pod ctest-pod-08133948 2024-11-15 01:22:31,323 - DEBUG - Pod : ctest-pod-08133948 UUID is 2ac9514b-8947-45e1-be1a-84ac0bbeca1a 2024-11-15 01:22:31,332 - DEBUG - Pod ctest-pod-21235123 not present 2024-11-15 01:22:31,332 - INFO - Creating Pod ctest-pod-21235123 2024-11-15 01:22:31,351 - DEBUG - Pod : ctest-pod-21235123 UUID is e8719320-7674-41de-94e2-860fb3003360 2024-11-15 01:22:31,353 - DEBUG - Pod ctest-pod-89314424 not present 2024-11-15 01:22:31,354 - INFO - Creating Pod ctest-pod-89314424 2024-11-15 01:22:31,371 - DEBUG - Pod : ctest-pod-89314424 UUID is 78963838-93f1-4d54-a156-03f70239ca7a 2024-11-15 01:22:31,374 - DEBUG - Pod ctest-pod-27916223 not present 2024-11-15 01:22:31,374 - INFO - Creating Pod ctest-pod-27916223 2024-11-15 01:22:31,391 - DEBUG - Pod : ctest-pod-27916223 UUID is 9bb82dc2-2168-44fe-a2a8-928d7b0f32c5 2024-11-15 01:22:31,394 - DEBUG - Pod ctest-pod-10317060 not present 2024-11-15 01:22:31,395 - INFO - Creating Pod ctest-pod-10317060 2024-11-15 01:22:31,414 - DEBUG - Pod : ctest-pod-10317060 UUID is 21122fc7-e888-48a7-be54-18cd5f7b68e4 2024-11-15 01:22:31,689 - INFO - Project default-domain:k8s-ctest-new-default-76912177 for Namespace ctest-new-default-76912177 is seen in contrail-api 2024-11-15 01:22:31,689 - DEBUG - Project uuid in Contrail is c149b82a-c196-44ed-8f69-e20c4b02e4eb 2024-11-15 01:22:31,859 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_NamespaceDatabaseList?namespace_uuid=f2e5319e-b3f7-43f4-a287-9fb06b288d7a 2024-11-15 01:22:31,891 - INFO - Namespace ctest-new-default-76912177 with uuid f2e5319e-b3f7-43f4-a287-9fb06b288d7a found in kube manager 2024-11-15 01:22:31,891 - INFO - Namespace ctest-new-default-76912177 verification passed 2024-11-15 01:22:32,132 - INFO - Project default-domain:k8s-ctest-non-default-80258725 for Namespace ctest-non-default-80258725 is seen in contrail-api 2024-11-15 01:22:32,132 - DEBUG - Project uuid in Contrail is 79f0db63-3621-443f-a760-73d935a7d036 2024-11-15 01:22:32,132 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_NamespaceDatabaseList?namespace_uuid=858b3dd7-7d13-4f4b-9fff-42a1648b445a 2024-11-15 01:22:32,137 - INFO - Namespace ctest-non-default-80258725 with uuid 858b3dd7-7d13-4f4b-9fff-42a1648b445a found in kube manager 2024-11-15 01:22:32,137 - INFO - Namespace ctest-non-default-80258725 verification passed 2024-11-15 01:22:32,146 - DEBUG - Pod ctest-pod-06545866 not in running state.Currently in Pending 2024-11-15 01:22:37,154 - DEBUG - Pod ctest-pod-06545866 not in running state.Currently in Pending 2024-11-15 01:22:42,160 - DEBUG - Pod ctest-pod-06545866 not in running state.Currently in Pending 2024-11-15 01:22:47,168 - INFO - Pod ctest-pod-06545866 is in running state.Got IP 10.47.255.251 2024-11-15 01:22:47,229 - DEBUG - Pod ctest-pod-06545866 has vmi 1719d912-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:47,230 - INFO - Verified pod ctest-pod-06545866 in contrail-api 2024-11-15 01:22:47,245 - DEBUG - VMI 1719d912-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:47,246 - INFO - Verified Pod ctest-pod-06545866 in agent 10.0.0.40 2024-11-15 01:22:47,246 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=5af10c86-9a51-4c0b-b120-893302f6dca8 2024-11-15 01:22:47,250 - INFO - Pod ctest-pod-06545866 with uuid 5af10c86-9a51-4c0b-b120-893302f6dca8 found in kube manager 2024-11-15 01:22:47,251 - INFO - Pod ctest-pod-06545866 verification passed 2024-11-15 01:22:47,256 - INFO - Pod ctest-pod-67580768 is in running state.Got IP 10.47.255.250 2024-11-15 01:22:47,275 - DEBUG - Pod ctest-pod-67580768 has vmi 17ad51f6-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:47,275 - INFO - Verified pod ctest-pod-67580768 in contrail-api 2024-11-15 01:22:47,289 - DEBUG - VMI 17ad51f6-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:47,289 - INFO - Verified Pod ctest-pod-67580768 in agent 10.0.0.40 2024-11-15 01:22:47,289 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=633375d7-57c6-4dc5-8275-3be3cd82ea7d 2024-11-15 01:22:47,294 - INFO - Pod ctest-pod-67580768 with uuid 633375d7-57c6-4dc5-8275-3be3cd82ea7d found in kube manager 2024-11-15 01:22:47,294 - INFO - Pod ctest-pod-67580768 verification passed 2024-11-15 01:22:47,301 - INFO - Pod ctest-pod-64402717 is in running state.Got IP 10.47.255.249 2024-11-15 01:22:47,362 - DEBUG - Pod ctest-pod-64402717 has vmi 185f5cac-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:47,362 - INFO - Verified pod ctest-pod-64402717 in contrail-api 2024-11-15 01:22:47,379 - DEBUG - VMI 185f5cac-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:47,379 - INFO - Verified Pod ctest-pod-64402717 in agent 10.0.0.40 2024-11-15 01:22:47,379 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=36d3a14b-ad9d-4c97-9d04-1b2df6ceae6e 2024-11-15 01:22:47,384 - INFO - Pod ctest-pod-64402717 with uuid 36d3a14b-ad9d-4c97-9d04-1b2df6ceae6e found in kube manager 2024-11-15 01:22:47,385 - INFO - Pod ctest-pod-64402717 verification passed 2024-11-15 01:22:47,390 - INFO - Pod ctest-pod-08133948 is in running state.Got IP 10.47.255.248 2024-11-15 01:22:47,462 - DEBUG - Pod ctest-pod-08133948 has vmi 18e3e486-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:47,462 - INFO - Verified pod ctest-pod-08133948 in contrail-api 2024-11-15 01:22:47,480 - DEBUG - VMI 18e3e486-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:47,481 - INFO - Verified Pod ctest-pod-08133948 in agent 10.0.0.40 2024-11-15 01:22:47,481 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=2ac9514b-8947-45e1-be1a-84ac0bbeca1a 2024-11-15 01:22:47,493 - INFO - Pod ctest-pod-08133948 with uuid 2ac9514b-8947-45e1-be1a-84ac0bbeca1a found in kube manager 2024-11-15 01:22:47,493 - INFO - Pod ctest-pod-08133948 verification passed 2024-11-15 01:22:47,499 - DEBUG - Pod ctest-pod-21235123 not in running state.Currently in Pending 2024-11-15 01:22:52,505 - INFO - Pod ctest-pod-21235123 is in running state.Got IP 10.47.255.247 2024-11-15 01:22:52,535 - DEBUG - Pod ctest-pod-21235123 has vmi 19792ca8-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:52,535 - INFO - Verified pod ctest-pod-21235123 in contrail-api 2024-11-15 01:22:52,548 - DEBUG - VMI 19792ca8-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:52,549 - INFO - Verified Pod ctest-pod-21235123 in agent 10.0.0.40 2024-11-15 01:22:52,549 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=e8719320-7674-41de-94e2-860fb3003360 2024-11-15 01:22:52,553 - INFO - Pod ctest-pod-21235123 with uuid e8719320-7674-41de-94e2-860fb3003360 found in kube manager 2024-11-15 01:22:52,553 - INFO - Pod ctest-pod-21235123 verification passed 2024-11-15 01:22:52,560 - INFO - Pod ctest-pod-27916223 is in running state.Got IP 10.47.255.245 2024-11-15 01:22:52,626 - DEBUG - Pod ctest-pod-27916223 has vmi 1a8528cc-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:52,626 - INFO - Verified pod ctest-pod-27916223 in contrail-api 2024-11-15 01:22:52,642 - DEBUG - VMI 1a8528cc-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:52,642 - INFO - Verified Pod ctest-pod-27916223 in agent 10.0.0.40 2024-11-15 01:22:52,642 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=9bb82dc2-2168-44fe-a2a8-928d7b0f32c5 2024-11-15 01:22:52,647 - INFO - Pod ctest-pod-27916223 with uuid 9bb82dc2-2168-44fe-a2a8-928d7b0f32c5 found in kube manager 2024-11-15 01:22:52,647 - INFO - Pod ctest-pod-27916223 verification passed 2024-11-15 01:22:52,653 - INFO - Pod ctest-pod-89314424 is in running state.Got IP 10.47.255.246 2024-11-15 01:22:52,671 - DEBUG - Pod ctest-pod-89314424 has vmi 19ff6610-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:52,671 - INFO - Verified pod ctest-pod-89314424 in contrail-api 2024-11-15 01:22:52,685 - DEBUG - VMI 19ff6610-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:52,685 - INFO - Verified Pod ctest-pod-89314424 in agent 10.0.0.40 2024-11-15 01:22:52,686 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=78963838-93f1-4d54-a156-03f70239ca7a 2024-11-15 01:22:52,699 - INFO - Pod ctest-pod-89314424 with uuid 78963838-93f1-4d54-a156-03f70239ca7a found in kube manager 2024-11-15 01:22:52,700 - INFO - Pod ctest-pod-89314424 verification passed 2024-11-15 01:22:52,707 - INFO - Pod ctest-pod-10317060 is in running state.Got IP 10.47.255.244 2024-11-15 01:22:52,775 - DEBUG - Pod ctest-pod-10317060 has vmi 1b03f7d8-a2f0-11ef-8635-fa163eb8ee38 2024-11-15 01:22:52,775 - INFO - Verified pod ctest-pod-10317060 in contrail-api 2024-11-15 01:22:52,806 - DEBUG - VMI 1b03f7d8-a2f0-11ef-8635-fa163eb8ee38 is active in agent 10.0.0.40 2024-11-15 01:22:52,806 - INFO - Verified Pod ctest-pod-10317060 in agent 10.0.0.40 2024-11-15 01:22:52,807 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_VirtualMachineDatabaseList?x=21122fc7-e888-48a7-be54-18cd5f7b68e4 2024-11-15 01:22:52,814 - INFO - Pod ctest-pod-10317060 with uuid 21122fc7-e888-48a7-be54-18cd5f7b68e4 found in kube manager 2024-11-15 01:22:52,815 - INFO - Pod ctest-pod-10317060 verification passed 2024-11-15 01:22:52,815 - INFO - ================================================================================ 2024-11-15 01:22:52,815 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-11-15 01:22:52,815 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-11-15 01:22:53,056 - DEBUG - Nothing to compare xmpp stats {'10.0.0.40': {'10.0.0.47': '0'}} with 2024-11-15 01:22:53,056 - INFO - Initial checks done. Running the testcase now 2024-11-15 01:22:53,056 - INFO - 2024-11-15 01:22:53,127 - DEBUG - [Pod ctest-pod-06545866] Cmd: echo ctest-pod-06545866 > /usr/share/nginx/html/index.html, Output: 2024-11-15 01:22:53,181 - DEBUG - [Pod ctest-pod-67580768] Cmd: echo ctest-pod-67580768 > /usr/share/nginx/html/index.html, Output: 2024-11-15 01:22:53,263 - DEBUG - [Pod ctest-pod-64402717] Cmd: echo ctest-pod-64402717 > /usr/share/nginx/html/index.html, Output: 2024-11-15 01:22:53,326 - DEBUG - [Pod ctest-pod-08133948] Cmd: echo ctest-pod-08133948 > /usr/share/nginx/html/index.html, Output: 2024-11-15 01:22:53,330 - DEBUG - Service ctest-nginx-svc-62233304 not present 2024-11-15 01:22:53,330 - INFO - Creating service ctest-nginx-svc-62233304 2024-11-15 01:22:53,345 - INFO - Created Service ctest-nginx-svc-62233304 2024-11-15 01:22:53,354 - WARNING - Service UUID 59ce039f-94bc-4745-8e47-8a5ff03dd45f not yet found in contrail-api 2024-11-15 01:22:54,371 - INFO - Validated that Service ctest-nginx-svc-62233304 is seen in contrail-api 2024-11-15 01:22:54,371 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_LoadbalancerDatabaseList?x=59ce039f-94bc-4745-8e47-8a5ff03dd45f 2024-11-15 01:22:54,378 - INFO - Service ctest-nginx-svc-62233304 with uuid 59ce039f-94bc-4745-8e47-8a5ff03dd45f found in kube manager 2024-11-15 01:22:54,378 - INFO - Service ctest-nginx-svc-62233304 verification passed 2024-11-15 01:22:55,466 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:22:55,466 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,466 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:22:55,536 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,536 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,536 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,604 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,604 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,604 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,671 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,671 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,671 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,742 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,742 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,742 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,742 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-06545866': 4, 'ctest-pod-67580768': 1} 2024-11-15 01:22:55,806 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,806 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,806 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:22:55,869 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:22:55,871 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:22:55,871 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:22:55,871 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-06545866': 1, 'ctest-pod-67580768': 1} 2024-11-15 01:22:55,876 - DEBUG - Network policy ingress-on-service not present 2024-11-15 01:22:55,876 - INFO - Creating Network Policy ingress-on-service 2024-11-15 01:22:55,890 - INFO - Network policy found in k8s 2024-11-15 01:22:55,890 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=59a566c2-71f6-4e18-aa48-e22a64b1196f 2024-11-15 01:22:55,895 - INFO - Network Policy ingress-on-service with uuid 59a566c2-71f6-4e18-aa48-e22a64b1196f found in kube manager 2024-11-15 01:22:55,928 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-11-15 01:22:55,929 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-11-15 01:22:55,929 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=59a566c2-71f6-4e18-aa48-e22a64b1196f 2024-11-15 01:22:55,974 - WARNING - Network policy with name ingress-on-service not found in agent 2024-11-15 01:22:57,974 - DEBUG - Requesting: http://10.0.0.47:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=59a566c2-71f6-4e18-aa48-e22a64b1196f 2024-11-15 01:22:58,007 - INFO - Network policy with name ingress-on-service found in agent 2024-11-15 01:22:58,008 - INFO - Network Policy ingress-on-service verification passed 2024-11-15 01:23:03,059 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:03,059 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:03,059 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:08,127 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:08,127 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:08,127 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:13,194 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:13,194 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:13,194 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:18,255 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:18,255 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:18,255 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:23,308 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:23,308 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:23,308 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:28,357 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:28,357 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:28,358 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:33,427 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:33,428 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:33,428 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:38,507 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:38,508 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:38,508 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:43,586 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:43,586 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:43,586 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:48,658 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:48,658 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:48,658 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:48,658 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-06545866': 0, 'ctest-pod-67580768': 0} 2024-11-15 01:23:48,719 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:23:48,719 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:23:48,719 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:23:48,777 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:23:48,777 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:23:48,777 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: ctest-pod-67580768 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:23:48,834 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:23:48,834 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:23:48,834 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:23:48,834 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-06545866': 1, 'ctest-pod-67580768': 2} 2024-11-15 01:23:53,898 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:53,898 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:53,898 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:58,962 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:58,963 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:23:58,963 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:04,015 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:04,015 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:04,015 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:09,091 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:09,091 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:09,091 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:14,163 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:14,163 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:14,163 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:16,581 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2024-11-15 01:24:26,682 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:26,682 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:26,682 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:31,746 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:31,746 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:31,746 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:36,820 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:36,820 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:36,821 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:41,881 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:41,881 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:41,881 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:46,954 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:46,954 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:46,954 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:24:46,955 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-06545866': 0, 'ctest-pod-67580768': 0} 2024-11-15 01:24:49,027 - DEBUG - [Pod ctest-pod-89314424] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.849 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.092 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.073 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.073/0.338/0.849 ms 2024-11-15 01:24:49,027 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-89314424 with expectation True passed 2024-11-15 01:24:51,075 - DEBUG - [Pod ctest-pod-21235123] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.490 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.082 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.087 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.082/0.219/0.490 ms 2024-11-15 01:24:51,075 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-21235123 with expectation True passed 2024-11-15 01:24:53,142 - DEBUG - [Pod ctest-pod-27916223] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=2.259 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.101 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.101 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.101/0.820/2.259 ms 2024-11-15 01:24:53,142 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-27916223 with expectation True passed 2024-11-15 01:24:53,210 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:24:53,210 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-15 01:24:53,210 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:24:53,210 - INFO - wget check of of http://10.47.255.251 from ctest-pod-89314424 passed 2024-11-15 01:24:53,283 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:24:53,283 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-11-15 01:24:53,284 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:24:53,284 - INFO - wget check of of http://10.47.255.250 from ctest-pod-89314424 passed 2024-11-15 01:24:58,350 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-15 01:24:58,350 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-15 01:24:58,350 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-15 01:24:58,350 - INFO - wget check of of http://10.47.255.250 from ctest-pod-21235123 passed 2024-11-15 01:25:03,407 - DEBUG - [Pod ctest-pod-10317060] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-15 01:25:03,407 - DEBUG - [Pod Pod ctest-pod-10317060] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-15 01:25:03,407 - DEBUG - [Pod Pod ctest-pod-10317060] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-15 01:25:03,408 - INFO - wget check of of http://10.47.255.251 from ctest-pod-10317060 passed 2024-11-15 01:25:03,416 - INFO - Updating Network Policy ingress-on-service 2024-11-15 01:25:08,484 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:08,485 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:08,485 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:13,550 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:13,550 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:13,550 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:18,624 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:18,624 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:18,624 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:23,677 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:23,677 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:23,677 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:28,746 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:28,746 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:28,746 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:33,806 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:33,806 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:33,806 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:38,872 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:38,872 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:38,872 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:43,950 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:43,951 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:43,951 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:49,014 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:49,014 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:49,014 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:54,063 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:54,063 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:54,063 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) wget: download timed out 2024-11-15 01:25:54,063 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-06545866': 0, 'ctest-pod-67580768': 0} 2024-11-15 01:25:54,122 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:25:54,123 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:25:54,123 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-06545866 2024-11-15 01:25:54,176 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:25:54,176 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:25:54,176 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:25:54,245 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-67580768 2024-11-15 01:25:54,245 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:25:54,245 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-67580768 2024-11-15 01:25:54,245 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-06545866': 2, 'ctest-pod-67580768': 1} 2024-11-15 01:25:54,319 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:25:54,319 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:25:54,319 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:25:54,387 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:25:54,388 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:25:54,388 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: ctest-pod-06545866 Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-15 01:25:54,439 - DEBUG - [Pod ctest-pod-27916223] Cmd: wget http://10.100.219.102:80/ -O - -T 5 -t 1 , Output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-67580768 2024-11-15 01:25:54,439 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd wget http://10.100.219.102:80/ -O - -T 5 -t 1 passed 2024-11-15 01:25:54,439 - DEBUG - [Pod Pod ctest-pod-27916223] Cmd output: Connecting to 10.100.219.102:80 (10.100.219.102:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-67580768 2024-11-15 01:25:54,440 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-06545866': 2, 'ctest-pod-67580768': 1} 2024-11-15 01:25:56,491 - DEBUG - [Pod ctest-pod-89314424] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.633 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.076 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.107 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.076/0.272/0.633 ms 2024-11-15 01:25:56,491 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-89314424 with expectation True passed 2024-11-15 01:25:58,544 - DEBUG - [Pod ctest-pod-21235123] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.587 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.073 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.086 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.073/0.248/0.587 ms 2024-11-15 01:25:58,544 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-21235123 with expectation True passed 2024-11-15 01:26:00,598 - DEBUG - [Pod ctest-pod-27916223] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.081 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.090 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.081 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.081/0.417/1.081 ms 2024-11-15 01:26:00,598 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-27916223 with expectation True passed 2024-11-15 01:26:00,663 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:26:00,663 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-15 01:26:00,663 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:26:00,663 - INFO - wget check of of http://10.47.255.251 from ctest-pod-89314424 passed 2024-11-15 01:26:00,713 - DEBUG - [Pod ctest-pod-89314424] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:26:00,713 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-11-15 01:26:00,713 - DEBUG - [Pod Pod ctest-pod-89314424] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:26:00,713 - INFO - wget check of of http://10.47.255.250 from ctest-pod-89314424 passed 2024-11-15 01:26:05,782 - DEBUG - [Pod ctest-pod-21235123] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-15 01:26:05,782 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-15 01:26:05,782 - DEBUG - [Pod Pod ctest-pod-21235123] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-15 01:26:05,782 - INFO - wget check of of http://10.47.255.250 from ctest-pod-21235123 passed 2024-11-15 01:26:05,858 - DEBUG - [Pod ctest-pod-10317060] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:26:05,858 - DEBUG - [Pod Pod ctest-pod-10317060] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-15 01:26:05,858 - DEBUG - [Pod Pod ctest-pod-10317060] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-15 01:26:05,858 - INFO - wget check of of http://10.47.255.251 from ctest-pod-10317060 passed 2024-11-15 01:26:05,858 - INFO - Deleting Network Policy : ingress-on-service 2024-11-15 01:26:05,858 - INFO - Deleting service : ctest-nginx-svc-62233304 2024-11-15 01:26:06,107 - DEBUG - No XMPP flaps were noticed during the test