2024-11-07 11:36:07,219 - INFO - Using existing project ['default-domain', 'k8s-default'](fce4ec1c-40ac-4845-be7b-4eb5bd46ba96) 2024-11-07 11:36:07,240 - DEBUG - Namespace ctest-new-default-72173396 not present 2024-11-07 11:36:07,248 - INFO - Created namespace ctest-new-default-72173396 2024-11-07 11:36:10,253 - DEBUG - Namespace ctest-non-default-93820247 not present 2024-11-07 11:36:10,259 - INFO - Created namespace ctest-non-default-93820247 2024-11-07 11:36:13,278 - DEBUG - Pod ctest-pod-36255665 not present 2024-11-07 11:36:13,279 - INFO - Creating Pod ctest-pod-36255665 2024-11-07 11:36:13,319 - DEBUG - Pod : ctest-pod-36255665 UUID is b2b8812f-13ee-4d48-8edd-1a23e1bdfc99 2024-11-07 11:36:13,334 - DEBUG - Pod ctest-pod-41343696 not present 2024-11-07 11:36:13,335 - INFO - Creating Pod ctest-pod-41343696 2024-11-07 11:36:13,366 - DEBUG - Pod : ctest-pod-41343696 UUID is 7606121c-6b0d-4981-b81d-f6539b7f6879 2024-11-07 11:36:13,368 - DEBUG - Pod ctest-pod-22248990 not present 2024-11-07 11:36:13,369 - INFO - Creating Pod ctest-pod-22248990 2024-11-07 11:36:13,396 - DEBUG - Pod : ctest-pod-22248990 UUID is 1186545c-5bdc-4dfa-a5a2-b7f56f63bdf8 2024-11-07 11:36:13,399 - DEBUG - Pod ctest-pod-33330458 not present 2024-11-07 11:36:13,399 - INFO - Creating Pod ctest-pod-33330458 2024-11-07 11:36:13,420 - DEBUG - Pod : ctest-pod-33330458 UUID is 010daeab-8951-444d-8321-7a500126ee39 2024-11-07 11:36:13,423 - DEBUG - Pod ctest-pod-35874862 not present 2024-11-07 11:36:13,423 - INFO - Creating Pod ctest-pod-35874862 2024-11-07 11:36:13,444 - DEBUG - Pod : ctest-pod-35874862 UUID is c5bc4960-f7c6-45a5-bbce-cb2c35f4b557 2024-11-07 11:36:13,446 - DEBUG - Pod ctest-pod-22253871 not present 2024-11-07 11:36:13,446 - INFO - Creating Pod ctest-pod-22253871 2024-11-07 11:36:13,474 - DEBUG - Pod : ctest-pod-22253871 UUID is 1b28773b-f152-4317-9599-d2666a417922 2024-11-07 11:36:13,476 - DEBUG - Pod ctest-pod-35796287 not present 2024-11-07 11:36:13,476 - INFO - Creating Pod ctest-pod-35796287 2024-11-07 11:36:13,494 - DEBUG - Pod : ctest-pod-35796287 UUID is 04c4cf45-11a1-47a4-8639-59f57910ff8d 2024-11-07 11:36:13,496 - DEBUG - Pod ctest-pod-59564476 not present 2024-11-07 11:36:13,496 - INFO - Creating Pod ctest-pod-59564476 2024-11-07 11:36:13,513 - DEBUG - Pod : ctest-pod-59564476 UUID is 9b1abeb0-9fdb-4e05-b0df-efb604d0c41f 2024-11-07 11:36:13,727 - INFO - Project default-domain:k8s-ctest-new-default-72173396 for Namespace ctest-new-default-72173396 is seen in contrail-api 2024-11-07 11:36:13,728 - DEBUG - Project uuid in Contrail is bc0420da-5e9f-4d8d-86e5-df434524ffeb 2024-11-07 11:36:13,882 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NamespaceDatabaseList?namespace_uuid=8bcc32e8-847c-4bac-bda4-97b0379b51d8 2024-11-07 11:36:13,902 - INFO - Namespace ctest-new-default-72173396 with uuid 8bcc32e8-847c-4bac-bda4-97b0379b51d8 found in kube manager 2024-11-07 11:36:13,902 - INFO - Namespace ctest-new-default-72173396 verification passed 2024-11-07 11:36:14,086 - INFO - Project default-domain:k8s-ctest-non-default-93820247 for Namespace ctest-non-default-93820247 is seen in contrail-api 2024-11-07 11:36:14,086 - DEBUG - Project uuid in Contrail is 55274ea6-91e6-40d1-9a85-68a4589442fa 2024-11-07 11:36:14,086 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NamespaceDatabaseList?namespace_uuid=f25e82ec-9ac8-4457-b5d6-c69f4eb83c0c 2024-11-07 11:36:14,091 - INFO - Namespace ctest-non-default-93820247 with uuid f25e82ec-9ac8-4457-b5d6-c69f4eb83c0c found in kube manager 2024-11-07 11:36:14,091 - INFO - Namespace ctest-non-default-93820247 verification passed 2024-11-07 11:36:14,096 - DEBUG - Pod ctest-pod-36255665 not in running state.Currently in Pending 2024-11-07 11:36:19,104 - DEBUG - Pod ctest-pod-36255665 not in running state.Currently in Pending 2024-11-07 11:36:24,113 - DEBUG - Pod ctest-pod-36255665 not in running state.Currently in Pending 2024-11-07 11:36:29,118 - INFO - Pod ctest-pod-36255665 is in running state.Got IP 10.47.255.251 2024-11-07 11:36:29,182 - DEBUG - Pod ctest-pod-36255665 has vmi 7f3fdc84-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:29,182 - INFO - Verified pod ctest-pod-36255665 in contrail-api 2024-11-07 11:36:29,198 - DEBUG - VMI 7f3fdc84-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:29,198 - INFO - Verified Pod ctest-pod-36255665 in agent 10.0.0.22 2024-11-07 11:36:29,198 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=b2b8812f-13ee-4d48-8edd-1a23e1bdfc99 2024-11-07 11:36:29,203 - INFO - Pod ctest-pod-36255665 with uuid b2b8812f-13ee-4d48-8edd-1a23e1bdfc99 found in kube manager 2024-11-07 11:36:29,203 - INFO - Pod ctest-pod-36255665 verification passed 2024-11-07 11:36:29,208 - INFO - Pod ctest-pod-41343696 is in running state.Got IP 10.47.255.250 2024-11-07 11:36:29,227 - DEBUG - Pod ctest-pod-41343696 has vmi 7fc0ca60-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:29,227 - INFO - Verified pod ctest-pod-41343696 in contrail-api 2024-11-07 11:36:29,242 - DEBUG - VMI 7fc0ca60-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:29,242 - INFO - Verified Pod ctest-pod-41343696 in agent 10.0.0.22 2024-11-07 11:36:29,242 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=7606121c-6b0d-4981-b81d-f6539b7f6879 2024-11-07 11:36:29,247 - INFO - Pod ctest-pod-41343696 with uuid 7606121c-6b0d-4981-b81d-f6539b7f6879 found in kube manager 2024-11-07 11:36:29,247 - INFO - Pod ctest-pod-41343696 verification passed 2024-11-07 11:36:29,253 - INFO - Pod ctest-pod-22248990 is in running state.Got IP 10.47.255.249 2024-11-07 11:36:29,326 - DEBUG - Pod ctest-pod-22248990 has vmi 8074d6c2-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:29,326 - INFO - Verified pod ctest-pod-22248990 in contrail-api 2024-11-07 11:36:29,338 - DEBUG - VMI 8074d6c2-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:29,339 - INFO - Verified Pod ctest-pod-22248990 in agent 10.0.0.22 2024-11-07 11:36:29,339 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=1186545c-5bdc-4dfa-a5a2-b7f56f63bdf8 2024-11-07 11:36:29,343 - INFO - Pod ctest-pod-22248990 with uuid 1186545c-5bdc-4dfa-a5a2-b7f56f63bdf8 found in kube manager 2024-11-07 11:36:29,344 - INFO - Pod ctest-pod-22248990 verification passed 2024-11-07 11:36:29,348 - INFO - Pod ctest-pod-33330458 is in running state.Got IP 10.47.255.248 2024-11-07 11:36:29,406 - DEBUG - Pod ctest-pod-33330458 has vmi 80f91734-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:29,406 - INFO - Verified pod ctest-pod-33330458 in contrail-api 2024-11-07 11:36:29,421 - DEBUG - VMI 80f91734-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:29,421 - INFO - Verified Pod ctest-pod-33330458 in agent 10.0.0.22 2024-11-07 11:36:29,421 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=010daeab-8951-444d-8321-7a500126ee39 2024-11-07 11:36:29,432 - INFO - Pod ctest-pod-33330458 with uuid 010daeab-8951-444d-8321-7a500126ee39 found in kube manager 2024-11-07 11:36:29,432 - INFO - Pod ctest-pod-33330458 verification passed 2024-11-07 11:36:29,439 - DEBUG - Pod ctest-pod-35874862 not in running state.Currently in Pending 2024-11-07 11:36:34,462 - INFO - Pod ctest-pod-35874862 is in running state.Got IP 10.47.255.247 2024-11-07 11:36:34,489 - DEBUG - Pod ctest-pod-35874862 has vmi 8194099c-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:34,489 - INFO - Verified pod ctest-pod-35874862 in contrail-api 2024-11-07 11:36:34,508 - DEBUG - VMI 8194099c-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:34,509 - INFO - Verified Pod ctest-pod-35874862 in agent 10.0.0.22 2024-11-07 11:36:34,509 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=c5bc4960-f7c6-45a5-bbce-cb2c35f4b557 2024-11-07 11:36:34,519 - INFO - Pod ctest-pod-35874862 with uuid c5bc4960-f7c6-45a5-bbce-cb2c35f4b557 found in kube manager 2024-11-07 11:36:34,520 - INFO - Pod ctest-pod-35874862 verification passed 2024-11-07 11:36:34,551 - INFO - Pod ctest-pod-35796287 is in running state.Got IP 10.47.255.245 2024-11-07 11:36:34,586 - DEBUG - Pod ctest-pod-35796287 has vmi 8299c5ac-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:34,587 - INFO - Verified pod ctest-pod-35796287 in contrail-api 2024-11-07 11:36:34,600 - DEBUG - VMI 8299c5ac-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:34,601 - INFO - Verified Pod ctest-pod-35796287 in agent 10.0.0.22 2024-11-07 11:36:34,601 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=04c4cf45-11a1-47a4-8639-59f57910ff8d 2024-11-07 11:36:34,609 - INFO - Pod ctest-pod-35796287 with uuid 04c4cf45-11a1-47a4-8639-59f57910ff8d found in kube manager 2024-11-07 11:36:34,611 - INFO - Pod ctest-pod-35796287 verification passed 2024-11-07 11:36:34,628 - INFO - Pod ctest-pod-22253871 is in running state.Got IP 10.47.255.246 2024-11-07 11:36:34,706 - DEBUG - Pod ctest-pod-22253871 has vmi 82183cee-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:34,706 - INFO - Verified pod ctest-pod-22253871 in contrail-api 2024-11-07 11:36:34,728 - DEBUG - VMI 82183cee-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:34,728 - INFO - Verified Pod ctest-pod-22253871 in agent 10.0.0.22 2024-11-07 11:36:34,728 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=1b28773b-f152-4317-9599-d2666a417922 2024-11-07 11:36:34,738 - INFO - Pod ctest-pod-22253871 with uuid 1b28773b-f152-4317-9599-d2666a417922 found in kube manager 2024-11-07 11:36:34,739 - INFO - Pod ctest-pod-22253871 verification passed 2024-11-07 11:36:34,745 - INFO - Pod ctest-pod-59564476 is in running state.Got IP 10.47.255.244 2024-11-07 11:36:34,765 - DEBUG - Pod ctest-pod-59564476 has vmi 831bad9c-9cfc-11ef-8286-fa163eb904e8 2024-11-07 11:36:34,765 - INFO - Verified pod ctest-pod-59564476 in contrail-api 2024-11-07 11:36:34,779 - DEBUG - VMI 831bad9c-9cfc-11ef-8286-fa163eb904e8 is active in agent 10.0.0.22 2024-11-07 11:36:34,779 - INFO - Verified Pod ctest-pod-59564476 in agent 10.0.0.22 2024-11-07 11:36:34,779 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=9b1abeb0-9fdb-4e05-b0df-efb604d0c41f 2024-11-07 11:36:34,797 - INFO - Pod ctest-pod-59564476 with uuid 9b1abeb0-9fdb-4e05-b0df-efb604d0c41f found in kube manager 2024-11-07 11:36:34,797 - INFO - Pod ctest-pod-59564476 verification passed 2024-11-07 11:36:34,798 - INFO - ================================================================================ 2024-11-07 11:36:34,798 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-11-07 11:36:34,799 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-11-07 11:36:35,037 - DEBUG - Nothing to compare xmpp stats {'10.0.0.22': {'10.0.0.34': '0'}} with 2024-11-07 11:36:35,038 - INFO - Initial checks done. Running the testcase now 2024-11-07 11:36:35,038 - INFO - 2024-11-07 11:36:35,110 - DEBUG - [Pod ctest-pod-36255665] Cmd: echo ctest-pod-36255665 > /usr/share/nginx/html/index.html, Output: 2024-11-07 11:36:35,178 - DEBUG - [Pod ctest-pod-41343696] Cmd: echo ctest-pod-41343696 > /usr/share/nginx/html/index.html, Output: 2024-11-07 11:36:35,236 - DEBUG - [Pod ctest-pod-22248990] Cmd: echo ctest-pod-22248990 > /usr/share/nginx/html/index.html, Output: 2024-11-07 11:36:35,296 - DEBUG - [Pod ctest-pod-33330458] Cmd: echo ctest-pod-33330458 > /usr/share/nginx/html/index.html, Output: 2024-11-07 11:36:35,299 - DEBUG - Service ctest-nginx-svc-83828330 not present 2024-11-07 11:36:35,299 - INFO - Creating service ctest-nginx-svc-83828330 2024-11-07 11:36:35,363 - INFO - Created Service ctest-nginx-svc-83828330 2024-11-07 11:36:35,383 - WARNING - Service UUID 8e92ebc1-c580-42b7-aa05-db119113d704 not yet found in contrail-api 2024-11-07 11:36:36,393 - INFO - Validated that Service ctest-nginx-svc-83828330 is seen in contrail-api 2024-11-07 11:36:36,394 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_LoadbalancerDatabaseList?x=8e92ebc1-c580-42b7-aa05-db119113d704 2024-11-07 11:36:36,404 - INFO - Service ctest-nginx-svc-83828330 with uuid 8e92ebc1-c580-42b7-aa05-db119113d704 found in kube manager 2024-11-07 11:36:36,404 - INFO - Service ctest-nginx-svc-83828330 verification passed 2024-11-07 11:36:37,494 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-36255665 2024-11-07 11:36:37,495 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:36:37,495 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-36255665 2024-11-07 11:36:37,568 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:36:37,568 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:36:37,568 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:36:37,642 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:36:37,642 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:36:37,642 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:36:37,642 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-36255665': 2, 'ctest-pod-41343696': 1} 2024-11-07 11:36:37,724 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:36:37,724 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:36:37,724 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:36:37,796 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:36:37,796 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:36:37,796 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:36:37,796 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-36255665': 1, 'ctest-pod-41343696': 1} 2024-11-07 11:36:37,799 - DEBUG - Network policy ingress-on-service not present 2024-11-07 11:36:37,800 - INFO - Creating Network Policy ingress-on-service 2024-11-07 11:36:37,811 - INFO - Network policy found in k8s 2024-11-07 11:36:37,812 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=86503dde-f575-4c82-aa9e-9187209ba7db 2024-11-07 11:36:37,821 - INFO - Network Policy ingress-on-service with uuid 86503dde-f575-4c82-aa9e-9187209ba7db found in kube manager 2024-11-07 11:36:37,855 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-11-07 11:36:37,856 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-11-07 11:36:37,856 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=86503dde-f575-4c82-aa9e-9187209ba7db 2024-11-07 11:36:37,893 - WARNING - Network policy with name ingress-on-service not found in agent 2024-11-07 11:36:39,894 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=86503dde-f575-4c82-aa9e-9187209ba7db 2024-11-07 11:36:39,926 - INFO - Network policy with name ingress-on-service found in agent 2024-11-07 11:36:39,927 - INFO - Network Policy ingress-on-service verification passed 2024-11-07 11:36:44,992 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:44,993 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:44,993 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:50,078 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:50,078 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:50,078 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:55,137 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:55,137 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:36:55,137 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:00,209 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:00,210 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:00,210 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:05,283 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:05,284 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:05,284 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:10,348 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:10,349 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:10,349 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:15,413 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:15,413 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:15,413 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:20,463 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:20,463 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:20,463 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:24,342 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2024-11-07 11:37:34,410 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:34,410 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:34,410 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:39,474 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:39,474 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:39,474 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:39,474 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-36255665': 0, 'ctest-pod-41343696': 0} 2024-11-07 11:37:39,539 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:37:39,539 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,540 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:37:39,594 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,594 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,594 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,659 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,659 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,659 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,733 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,733 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,733 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,799 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,799 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,799 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,869 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:37:39,870 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,870 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:37:39,946 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,946 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:37:39,946 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:37:39,946 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-36255665': 1, 'ctest-pod-41343696': 6} 2024-11-07 11:37:45,026 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:45,026 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:45,026 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:50,095 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:50,096 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:50,096 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:55,160 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:55,160 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:37:55,160 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:00,216 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:00,216 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:00,216 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:05,282 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:05,282 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:05,282 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:10,338 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:10,338 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:10,338 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:15,389 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:15,389 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:15,389 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:20,456 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:20,456 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:20,456 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:25,532 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:25,532 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:25,532 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:30,601 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:30,601 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:30,601 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:30,601 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-36255665': 0, 'ctest-pod-41343696': 0} 2024-11-07 11:38:32,671 - DEBUG - [Pod ctest-pod-22253871] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.496 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.072 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.079 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.072/0.215/0.496 ms 2024-11-07 11:38:32,671 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-22253871 with expectation True passed 2024-11-07 11:38:34,728 - DEBUG - [Pod ctest-pod-35874862] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.326 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.073 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.100 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.073/0.499/1.326 ms 2024-11-07 11:38:34,728 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-35874862 with expectation True passed 2024-11-07 11:38:36,799 - DEBUG - [Pod ctest-pod-35796287] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.532 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.115 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.068 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.068/0.238/0.532 ms 2024-11-07 11:38:36,799 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-35796287 with expectation True passed 2024-11-07 11:38:36,863 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:38:36,863 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-07 11:38:36,863 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:38:36,863 - INFO - wget check of of http://10.47.255.251 from ctest-pod-22253871 passed 2024-11-07 11:38:36,925 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:38:36,925 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-11-07 11:38:36,925 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:38:36,925 - INFO - wget check of of http://10.47.255.250 from ctest-pod-22253871 passed 2024-11-07 11:38:41,987 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-07 11:38:41,988 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-07 11:38:41,988 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-07 11:38:41,988 - INFO - wget check of of http://10.47.255.250 from ctest-pod-35874862 passed 2024-11-07 11:38:47,057 - DEBUG - [Pod ctest-pod-59564476] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-07 11:38:47,058 - DEBUG - [Pod Pod ctest-pod-59564476] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-07 11:38:47,058 - DEBUG - [Pod Pod ctest-pod-59564476] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-07 11:38:47,058 - INFO - wget check of of http://10.47.255.251 from ctest-pod-59564476 passed 2024-11-07 11:38:47,066 - INFO - Updating Network Policy ingress-on-service 2024-11-07 11:38:52,124 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:52,124 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:52,124 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:57,185 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:57,186 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:38:57,186 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:02,261 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:02,261 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:02,261 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:07,325 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:07,326 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:07,326 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:12,393 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:12,393 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:12,393 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:17,466 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:17,466 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:17,466 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:22,518 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:22,518 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:22,518 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:27,579 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:27,580 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:27,580 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:32,637 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:32,637 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:32,637 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:37,718 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:37,718 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:37,718 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) wget: download timed out 2024-11-07 11:39:37,718 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-36255665': 0, 'ctest-pod-41343696': 0} 2024-11-07 11:39:37,782 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,782 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:37,782 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,848 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,848 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:37,849 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: ctest-pod-41343696 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,849 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-36255665': 1, 'ctest-pod-41343696': 1} 2024-11-07 11:39:37,921 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,921 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:37,921 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,981 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:37,981 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:37,981 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:38,032 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:38,032 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:38,032 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:38,104 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:38,104 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:38,104 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: ctest-pod-36255665 Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-07 11:39:38,178 - DEBUG - [Pod ctest-pod-35796287] Cmd: wget http://10.103.35.118:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:39:38,178 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd wget http://10.103.35.118:80/ -O - -T 5 -t 1 passed 2024-11-07 11:39:38,178 - DEBUG - [Pod Pod ctest-pod-35796287] Cmd output: Connecting to 10.103.35.118:80 (10.103.35.118:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41343696 2024-11-07 11:39:38,178 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-36255665': 4, 'ctest-pod-41343696': 1} 2024-11-07 11:39:40,240 - DEBUG - [Pod ctest-pod-22253871] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.473 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.063 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.080 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.063/0.538/1.473 ms 2024-11-07 11:39:40,240 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-22253871 with expectation True passed 2024-11-07 11:39:42,297 - DEBUG - [Pod ctest-pod-35874862] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.986 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.093 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.101 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.093/0.393/0.986 ms 2024-11-07 11:39:42,297 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-35874862 with expectation True passed 2024-11-07 11:39:44,375 - DEBUG - [Pod ctest-pod-35796287] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.708 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.082 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.080 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.080/0.290/0.708 ms 2024-11-07 11:39:44,406 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-35796287 with expectation True passed 2024-11-07 11:39:44,462 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:39:44,462 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-07 11:39:44,462 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:39:44,462 - INFO - wget check of of http://10.47.255.251 from ctest-pod-22253871 passed 2024-11-07 11:39:44,533 - DEBUG - [Pod ctest-pod-22253871] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:39:44,533 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-11-07 11:39:44,533 - DEBUG - [Pod Pod ctest-pod-22253871] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:39:44,533 - INFO - wget check of of http://10.47.255.250 from ctest-pod-22253871 passed 2024-11-07 11:39:49,596 - DEBUG - [Pod ctest-pod-35874862] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-07 11:39:49,596 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-07 11:39:49,596 - DEBUG - [Pod Pod ctest-pod-35874862] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-07 11:39:49,596 - INFO - wget check of of http://10.47.255.250 from ctest-pod-35874862 passed 2024-11-07 11:39:49,665 - DEBUG - [Pod ctest-pod-59564476] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:39:49,665 - DEBUG - [Pod Pod ctest-pod-59564476] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-07 11:39:49,665 - DEBUG - [Pod Pod ctest-pod-59564476] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-07 11:39:49,665 - INFO - wget check of of http://10.47.255.251 from ctest-pod-59564476 passed 2024-11-07 11:39:49,666 - INFO - Deleting Network Policy : ingress-on-service 2024-11-07 11:39:49,666 - INFO - Deleting service : ctest-nginx-svc-83828330 2024-11-07 11:39:50,067 - DEBUG - No XMPP flaps were noticed during the test