2025-01-06 02:03:07,390 - INFO - Using existing project ['default-domain', 'k8s-default'](912c94ea-0fd0-40e2-a8ad-b1a985b59c9e)
2025-01-06 02:03:07,416 - DEBUG - Namespace ctest-new-default-04656451 not present
2025-01-06 02:03:07,423 - INFO - Created namespace ctest-new-default-04656451
2025-01-06 02:03:10,427 - DEBUG - Namespace ctest-non-default-63389887 not present
2025-01-06 02:03:10,442 - INFO - Created namespace ctest-non-default-63389887
2025-01-06 02:03:13,456 - DEBUG - Pod ctest-pod-28190471 not present
2025-01-06 02:03:13,457 - INFO - Creating Pod ctest-pod-28190471
2025-01-06 02:03:13,485 - DEBUG - Pod : ctest-pod-28190471 UUID is cdac6568-6098-4916-8c4d-d9991f2ecb06
2025-01-06 02:03:13,495 - DEBUG - Pod ctest-pod-89480523 not present
2025-01-06 02:03:13,496 - INFO - Creating Pod ctest-pod-89480523
2025-01-06 02:03:13,518 - DEBUG - Pod : ctest-pod-89480523 UUID is 546ea400-226f-4b47-a0d9-2845c485bf10
2025-01-06 02:03:13,522 - DEBUG - Pod ctest-pod-83273086 not present
2025-01-06 02:03:13,522 - INFO - Creating Pod ctest-pod-83273086
2025-01-06 02:03:13,563 - DEBUG - Pod : ctest-pod-83273086 UUID is f53c16bd-a4a0-4c25-b513-d721b71cffcb
2025-01-06 02:03:13,565 - DEBUG - Pod ctest-pod-33005488 not present
2025-01-06 02:03:13,565 - INFO - Creating Pod ctest-pod-33005488
2025-01-06 02:03:13,584 - DEBUG - Pod : ctest-pod-33005488 UUID is c14bd24b-7442-47ae-aa32-bba565ee7d4d
2025-01-06 02:03:13,586 - DEBUG - Pod ctest-pod-50743923 not present
2025-01-06 02:03:13,586 - INFO - Creating Pod ctest-pod-50743923
2025-01-06 02:03:13,601 - DEBUG - Pod : ctest-pod-50743923 UUID is 8d59364e-3d39-450f-a2e0-ac6e4a709b0b
2025-01-06 02:03:13,603 - DEBUG - Pod ctest-pod-86431628 not present
2025-01-06 02:03:13,603 - INFO - Creating Pod ctest-pod-86431628
2025-01-06 02:03:13,619 - DEBUG - Pod : ctest-pod-86431628 UUID is 6213f40a-c69c-45b6-9c83-2b17e0f398b3
2025-01-06 02:03:13,622 - DEBUG - Pod ctest-pod-29627795 not present
2025-01-06 02:03:13,622 - INFO - Creating Pod ctest-pod-29627795
2025-01-06 02:03:13,638 - DEBUG - Pod : ctest-pod-29627795 UUID is f2b96afc-18b5-415a-b1f3-6de39f914229
2025-01-06 02:03:13,640 - DEBUG - Pod ctest-pod-43910232 not present
2025-01-06 02:03:13,640 - INFO - Creating Pod ctest-pod-43910232
2025-01-06 02:03:13,660 - DEBUG - Pod : ctest-pod-43910232 UUID is ca363076-2cdb-4e99-9fb7-bbf9fc49464d
2025-01-06 02:03:13,819 - INFO - Project default-domain:k8s-ctest-new-default-04656451 for Namespace ctest-new-default-04656451 is seen in contrail-api
2025-01-06 02:03:13,819 - DEBUG - Project uuid in Contrail is 9573ddd1-bede-4e95-a4d3-c83c25ced8ae
2025-01-06 02:03:14,126 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NamespaceDatabaseList?namespace_uuid=317940e4-ec15-4ae9-8f44-77a73b8ac3ff
2025-01-06 02:03:14,130 - INFO - Namespace ctest-new-default-04656451 with uuid 317940e4-ec15-4ae9-8f44-77a73b8ac3ff found in kube manager
2025-01-06 02:03:14,130 - INFO - Namespace ctest-new-default-04656451 verification passed
2025-01-06 02:03:14,306 - INFO - Project default-domain:k8s-ctest-non-default-63389887 for Namespace ctest-non-default-63389887 is seen in contrail-api
2025-01-06 02:03:14,306 - DEBUG - Project uuid in Contrail is 049d84fe-37fd-467f-9299-e07796339f67
2025-01-06 02:03:14,306 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NamespaceDatabaseList?namespace_uuid=857d33f8-3fd4-496c-98b3-0b3a6a899e69
2025-01-06 02:03:14,310 - INFO - Namespace ctest-non-default-63389887 with uuid 857d33f8-3fd4-496c-98b3-0b3a6a899e69 found in kube manager
2025-01-06 02:03:14,310 - INFO - Namespace ctest-non-default-63389887 verification passed
2025-01-06 02:03:14,316 - DEBUG - Pod ctest-pod-28190471 not in running state.Currently in Pending
2025-01-06 02:03:19,331 - DEBUG - Pod ctest-pod-28190471 not in running state.Currently in Pending
2025-01-06 02:03:24,338 - INFO - Pod ctest-pod-28190471 is in running state.Got IP 10.47.255.251
2025-01-06 02:03:24,394 - DEBUG - Pod ctest-pod-28190471 has vmi 63e2b830-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:24,394 - INFO - Verified pod ctest-pod-28190471 in contrail-api
2025-01-06 02:03:24,406 - DEBUG - VMI 63e2b830-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:24,407 - INFO - Verified Pod ctest-pod-28190471 in agent 10.0.0.34
2025-01-06 02:03:24,407 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=cdac6568-6098-4916-8c4d-d9991f2ecb06
2025-01-06 02:03:24,411 - INFO - Pod ctest-pod-28190471 with uuid cdac6568-6098-4916-8c4d-d9991f2ecb06 found in kube manager
2025-01-06 02:03:24,411 - INFO - Pod ctest-pod-28190471 verification passed
2025-01-06 02:03:24,416 - INFO - Pod ctest-pod-89480523 is in running state.Got IP 10.47.255.250
2025-01-06 02:03:24,429 - DEBUG - Pod ctest-pod-89480523 has vmi 645aac8c-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:24,429 - INFO - Verified pod ctest-pod-89480523 in contrail-api
2025-01-06 02:03:24,443 - DEBUG - VMI 645aac8c-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:24,443 - INFO - Verified Pod ctest-pod-89480523 in agent 10.0.0.34
2025-01-06 02:03:24,443 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=546ea400-226f-4b47-a0d9-2845c485bf10
2025-01-06 02:03:24,447 - INFO - Pod ctest-pod-89480523 with uuid 546ea400-226f-4b47-a0d9-2845c485bf10 found in kube manager
2025-01-06 02:03:24,447 - INFO - Pod ctest-pod-89480523 verification passed
2025-01-06 02:03:24,452 - INFO - Pod ctest-pod-83273086 is in running state.Got IP 10.47.255.249
2025-01-06 02:03:24,515 - DEBUG - Pod ctest-pod-83273086 has vmi 64faa9c6-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:24,515 - INFO - Verified pod ctest-pod-83273086 in contrail-api
2025-01-06 02:03:24,527 - DEBUG - VMI 64faa9c6-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:24,527 - INFO - Verified Pod ctest-pod-83273086 in agent 10.0.0.34
2025-01-06 02:03:24,527 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=f53c16bd-a4a0-4c25-b513-d721b71cffcb
2025-01-06 02:03:24,531 - INFO - Pod ctest-pod-83273086 with uuid f53c16bd-a4a0-4c25-b513-d721b71cffcb found in kube manager
2025-01-06 02:03:24,531 - INFO - Pod ctest-pod-83273086 verification passed
2025-01-06 02:03:24,537 - DEBUG - Pod ctest-pod-33005488 not in running state.Currently in Pending
2025-01-06 02:03:29,545 - INFO - Pod ctest-pod-33005488 is in running state.Got IP 10.47.255.248
2025-01-06 02:03:29,561 - DEBUG - Pod ctest-pod-33005488 has vmi 6577f034-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:29,562 - INFO - Verified pod ctest-pod-33005488 in contrail-api
2025-01-06 02:03:29,576 - DEBUG - VMI 6577f034-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:29,577 - INFO - Verified Pod ctest-pod-33005488 in agent 10.0.0.34
2025-01-06 02:03:29,577 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=c14bd24b-7442-47ae-aa32-bba565ee7d4d
2025-01-06 02:03:29,581 - INFO - Pod ctest-pod-33005488 with uuid c14bd24b-7442-47ae-aa32-bba565ee7d4d found in kube manager
2025-01-06 02:03:29,581 - INFO - Pod ctest-pod-33005488 verification passed
2025-01-06 02:03:29,586 - DEBUG - Pod ctest-pod-50743923 not in running state.Currently in Pending
2025-01-06 02:03:34,593 - INFO - Pod ctest-pod-50743923 is in running state.Got IP 10.47.255.247
2025-01-06 02:03:34,646 - DEBUG - Pod ctest-pod-50743923 has vmi 6602ffd0-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:34,646 - INFO - Verified pod ctest-pod-50743923 in contrail-api
2025-01-06 02:03:34,660 - DEBUG - VMI 6602ffd0-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:34,661 - INFO - Verified Pod ctest-pod-50743923 in agent 10.0.0.34
2025-01-06 02:03:34,661 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=8d59364e-3d39-450f-a2e0-ac6e4a709b0b
2025-01-06 02:03:34,665 - INFO - Pod ctest-pod-50743923 with uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b found in kube manager
2025-01-06 02:03:34,665 - INFO - Pod ctest-pod-50743923 verification passed
2025-01-06 02:03:34,669 - INFO - Pod ctest-pod-29627795 is in running state.Got IP 10.47.255.245
2025-01-06 02:03:34,682 - DEBUG - Pod ctest-pod-29627795 has vmi 66ef5416-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:34,682 - INFO - Verified pod ctest-pod-29627795 in contrail-api
2025-01-06 02:03:34,696 - DEBUG - VMI 66ef5416-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:34,696 - INFO - Verified Pod ctest-pod-29627795 in agent 10.0.0.34
2025-01-06 02:03:34,697 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=f2b96afc-18b5-415a-b1f3-6de39f914229
2025-01-06 02:03:34,701 - INFO - Pod ctest-pod-29627795 with uuid f2b96afc-18b5-415a-b1f3-6de39f914229 found in kube manager
2025-01-06 02:03:34,701 - INFO - Pod ctest-pod-29627795 verification passed
2025-01-06 02:03:34,706 - INFO - Pod ctest-pod-86431628 is in running state.Got IP 10.47.255.246
2025-01-06 02:03:34,763 - DEBUG - Pod ctest-pod-86431628 has vmi 667c8bd4-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:34,763 - INFO - Verified pod ctest-pod-86431628 in contrail-api
2025-01-06 02:03:34,778 - DEBUG - VMI 667c8bd4-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:34,778 - INFO - Verified Pod ctest-pod-86431628 in agent 10.0.0.34
2025-01-06 02:03:34,778 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=6213f40a-c69c-45b6-9c83-2b17e0f398b3
2025-01-06 02:03:34,783 - INFO - Pod ctest-pod-86431628 with uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 found in kube manager
2025-01-06 02:03:34,783 - INFO - Pod ctest-pod-86431628 verification passed
2025-01-06 02:03:34,788 - INFO - Pod ctest-pod-43910232 is in running state.Got IP 10.47.255.244
2025-01-06 02:03:34,846 - DEBUG - Pod ctest-pod-43910232 has vmi 676706c8-cbd2-11ef-b954-fa163e6f847a
2025-01-06 02:03:34,846 - INFO - Verified pod ctest-pod-43910232 in contrail-api
2025-01-06 02:03:34,860 - DEBUG - VMI 676706c8-cbd2-11ef-b954-fa163e6f847a is active in agent 10.0.0.34
2025-01-06 02:03:34,861 - INFO - Verified Pod ctest-pod-43910232 in agent 10.0.0.34
2025-01-06 02:03:34,861 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=ca363076-2cdb-4e99-9fb7-bbf9fc49464d
2025-01-06 02:03:34,865 - INFO - Pod ctest-pod-43910232 with uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d found in kube manager
2025-01-06 02:03:34,865 - INFO - Pod ctest-pod-43910232 verification passed
2025-01-06 02:03:34,865 - INFO - ================================================================================
2025-01-06 02:03:34,865 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service
2025-01-06 02:03:34,865 - INFO - TEST DESCRIPTION :
Verify that ingress rules on a pod work fine when pods are accessed via a service
Steps:
1. Create a service in namespace 1 over a pod which are endpoints of a service
2. Create a network policy to allow 1 pod of namespace 1 as ingress.
3. Verify that service respond correctly to the specified pod in the policy and
service do not respond to anyone else.
4. Extend the policy and add the namespaceselector as namespace 2.
5. Verify that service become accessible to the pods of namespace 2.
2025-01-06 02:03:35,106 - DEBUG - Nothing to compare xmpp stats {'10.0.0.34': {'10.0.0.28': '0'}} with
2025-01-06 02:03:35,107 - INFO - Initial checks done. Running the testcase now
2025-01-06 02:03:35,107 - INFO -
2025-01-06 02:03:35,161 - DEBUG - [Pod ctest-pod-28190471] Cmd: echo ctest-pod-28190471 > /usr/share/nginx/html/index.html, Output:
2025-01-06 02:03:35,229 - DEBUG - [Pod ctest-pod-89480523] Cmd: echo ctest-pod-89480523 > /usr/share/nginx/html/index.html, Output:
2025-01-06 02:03:35,292 - DEBUG - [Pod ctest-pod-83273086] Cmd: echo ctest-pod-83273086 > /usr/share/nginx/html/index.html, Output:
2025-01-06 02:03:35,369 - DEBUG - [Pod ctest-pod-33005488] Cmd: echo ctest-pod-33005488 > /usr/share/nginx/html/index.html, Output:
2025-01-06 02:03:35,372 - DEBUG - Service ctest-nginx-svc-11022438 not present
2025-01-06 02:03:35,372 - INFO - Creating service ctest-nginx-svc-11022438
2025-01-06 02:03:35,387 - INFO - Created Service ctest-nginx-svc-11022438
2025-01-06 02:03:35,402 - WARNING - Service UUID cdebd6a2-769d-4470-ae08-b95077fa7e00 not yet found in contrail-api
2025-01-06 02:03:36,440 - INFO - Validated that Service ctest-nginx-svc-11022438 is seen in contrail-api
2025-01-06 02:03:36,441 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_LoadbalancerDatabaseList?x=cdebd6a2-769d-4470-ae08-b95077fa7e00
2025-01-06 02:03:36,446 - INFO - Service ctest-nginx-svc-11022438 with uuid cdebd6a2-769d-4470-ae08-b95077fa7e00 found in kube manager
2025-01-06 02:03:36,446 - INFO - Service ctest-nginx-svc-11022438 verification passed
2025-01-06 02:03:37,544 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
ctest-pod-28190471
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,544 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,544 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
ctest-pod-28190471
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,615 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-28190471
2025-01-06 02:03:37,615 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,615 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
ctest-pod-28190471
2025-01-06 02:03:37,677 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,677 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,677 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,748 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,748 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,748 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,819 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,819 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,819 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,819 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-28190471': 4, 'ctest-pod-89480523': 1}
2025-01-06 02:03:37,895 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,895 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,895 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,955 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,956 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:03:37,956 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:03:37,956 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-28190471': 1, 'ctest-pod-89480523': 1}
2025-01-06 02:03:37,959 - DEBUG - Network policy ingress-on-service not present
2025-01-06 02:03:37,959 - INFO - Creating Network Policy ingress-on-service
2025-01-06 02:03:37,973 - INFO - Network policy found in k8s
2025-01-06 02:03:37,973 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=f99d0749-0ee4-4fcc-b17a-6bf2a19ed31a
2025-01-06 02:03:37,978 - INFO - Network Policy ingress-on-service with uuid f99d0749-0ee4-4fcc-b17a-6bf2a19ed31a found in kube manager
2025-01-06 02:03:38,012 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent
2025-01-06 02:03:38,013 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8
2025-01-06 02:03:38,013 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=f99d0749-0ee4-4fcc-b17a-6bf2a19ed31a
2025-01-06 02:03:38,043 - WARNING - Network policy with name ingress-on-service not found in agent
2025-01-06 02:03:40,044 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=f99d0749-0ee4-4fcc-b17a-6bf2a19ed31a
2025-01-06 02:03:40,077 - INFO - Network policy with name ingress-on-service found in agent
2025-01-06 02:03:40,077 - INFO - Network Policy ingress-on-service verification passed
2025-01-06 02:03:45,156 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:45,156 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:45,156 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:50,242 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:50,242 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:50,242 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:55,311 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:55,311 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:03:55,311 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:00,384 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:00,384 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:00,384 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:05,448 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:05,448 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:05,448 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:10,506 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:10,506 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:10,506 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:15,564 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:15,564 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:15,564 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:20,634 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:20,634 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:20,634 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:25,705 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:25,705 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:25,705 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:30,785 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:30,786 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:30,786 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:30,786 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-28190471': 0, 'ctest-pod-89480523': 0}
2025-01-06 02:04:30,865 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:04:30,865 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:04:30,865 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:04:30,924 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:04:30,924 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:04:30,924 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:04:30,924 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-28190471': 1, 'ctest-pod-89480523': 1}
2025-01-06 02:04:35,998 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:35,998 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:35,998 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:41,049 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:41,049 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:41,050 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:46,099 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:46,099 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:46,099 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:51,166 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:51,166 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:51,166 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:56,218 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:56,218 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:04:56,218 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:01,279 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:01,280 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:01,280 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:06,334 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:06,334 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:06,334 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:11,394 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:11,394 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:11,394 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:16,480 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:16,480 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:16,480 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:21,543 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:21,543 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:21,543 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:21,543 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-28190471': 0, 'ctest-pod-89480523': 0}
2025-01-06 02:05:23,619 - DEBUG - [Pod ctest-pod-86431628] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes
64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.422 ms
64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.101 ms
64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.098 ms
--- 10.47.255.247 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.098/0.540/1.422 ms
2025-01-06 02:05:23,619 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-86431628 with expectation True passed
2025-01-06 02:05:25,678 - DEBUG - [Pod ctest-pod-50743923] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.957 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.101 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.091 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.091/0.383/0.957 ms
2025-01-06 02:05:25,678 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-50743923 with expectation True passed
2025-01-06 02:05:27,729 - DEBUG - [Pod ctest-pod-29627795] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=2.019 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.079 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.086 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.079/0.728/2.019 ms
2025-01-06 02:05:27,729 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-29627795 with expectation True passed
2025-01-06 02:05:27,795 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:05:27,795 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed
2025-01-06 02:05:27,795 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:05:27,795 - INFO - wget check of of http://10.47.255.251 from ctest-pod-86431628 passed
2025-01-06 02:05:27,878 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:05:27,878 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed
2025-01-06 02:05:27,878 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:05:27,878 - INFO - wget check of of http://10.47.255.250 from ctest-pod-86431628 passed
2025-01-06 02:05:32,958 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-06 02:05:32,958 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-06 02:05:32,958 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-06 02:05:32,958 - INFO - wget check of of http://10.47.255.250 from ctest-pod-50743923 passed
2025-01-06 02:05:38,022 - DEBUG - [Pod ctest-pod-43910232] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
wget: download timed out
2025-01-06 02:05:38,022 - DEBUG - [Pod Pod ctest-pod-43910232] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80)
wget: download timed out
2025-01-06 02:05:38,022 - DEBUG - [Pod Pod ctest-pod-43910232] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
wget: download timed out
2025-01-06 02:05:38,022 - INFO - wget check of of http://10.47.255.251 from ctest-pod-43910232 passed
2025-01-06 02:05:38,030 - INFO - Updating Network Policy ingress-on-service
2025-01-06 02:05:43,103 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:43,103 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:43,103 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:48,169 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:48,169 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:48,170 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:53,232 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:53,232 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:53,232 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:58,289 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:58,290 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:05:58,290 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:03,353 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:03,353 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:03,353 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:08,417 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:08,417 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:08,418 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:13,488 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:13,489 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:13,489 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:18,558 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:18,558 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:18,559 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:23,648 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:23,648 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:23,648 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:26,014 - WARNING - ApiException is caught (0)
Reason: Connection is already closed.
. Retrying...
2025-01-06 02:06:36,463 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:36,463 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:36,463 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
wget: download timed out
2025-01-06 02:06:36,463 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-28190471': 0, 'ctest-pod-89480523': 0}
2025-01-06 02:06:36,530 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,530 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:06:36,530 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,604 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,604 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:06:36,604 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,673 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,673 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:06:36,673 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: ctest-pod-89480523
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,734 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,734 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:06:36,734 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,734 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-28190471': 1, 'ctest-pod-89480523': 3}
2025-01-06 02:06:36,789 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
ctest-pod-89480523
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,789 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:06:36,789 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
ctest-pod-89480523
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,843 - DEBUG - [Pod ctest-pod-29627795] Cmd: wget http://10.111.234.149:80/ -O - -T 5 -t 1 , Output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,844 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd wget http://10.111.234.149:80/ -O - -T 5 -t 1 passed
2025-01-06 02:06:36,844 - DEBUG - [Pod Pod ctest-pod-29627795] Cmd output: ctest-pod-28190471
Connecting to 10.111.234.149:80 (10.111.234.149:80)
writing to stdout
- 100% |********************************| 19 0:00:00 ETA
written to stdout
2025-01-06 02:06:36,844 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-28190471': 1, 'ctest-pod-89480523': 1}
2025-01-06 02:06:38,898 - DEBUG - [Pod ctest-pod-86431628] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes
64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.455 ms
64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.081 ms
64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.104 ms
--- 10.47.255.247 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.081/0.213/0.455 ms
2025-01-06 02:06:38,898 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-86431628 with expectation True passed
2025-01-06 02:06:40,967 - DEBUG - [Pod ctest-pod-50743923] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.024 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.079 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.104 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.079/0.402/1.024 ms
2025-01-06 02:06:40,967 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-50743923 with expectation True passed
2025-01-06 02:06:43,050 - DEBUG - [Pod ctest-pod-29627795] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes
64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.421 ms
64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.119 ms
64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.075 ms
--- 10.47.255.246 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.075/0.205/0.421 ms
2025-01-06 02:06:43,050 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-29627795 with expectation True passed
2025-01-06 02:06:43,116 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:06:43,117 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed
2025-01-06 02:06:43,117 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:06:43,117 - INFO - wget check of of http://10.47.255.251 from ctest-pod-86431628 passed
2025-01-06 02:06:43,199 - DEBUG - [Pod ctest-pod-86431628] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:06:43,199 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed
2025-01-06 02:06:43,199 - DEBUG - [Pod Pod ctest-pod-86431628] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:06:43,199 - INFO - wget check of of http://10.47.255.250 from ctest-pod-86431628 passed
2025-01-06 02:06:48,270 - DEBUG - [Pod ctest-pod-50743923] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-06 02:06:48,270 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-06 02:06:48,270 - DEBUG - [Pod Pod ctest-pod-50743923] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80)
wget: download timed out
2025-01-06 02:06:48,270 - INFO - wget check of of http://10.47.255.250 from ctest-pod-50743923 passed
2025-01-06 02:06:48,342 - DEBUG - [Pod ctest-pod-43910232] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:06:48,342 - DEBUG - [Pod Pod ctest-pod-43910232] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed
2025-01-06 02:06:48,342 - DEBUG - [Pod Pod ctest-pod-43910232] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80)
saving to '/dev/null'
null 100% |********************************| 19 0:00:00 ETA
'/dev/null' saved
2025-01-06 02:06:48,342 - INFO - wget check of of http://10.47.255.251 from ctest-pod-43910232 passed
2025-01-06 02:06:48,342 - INFO - Deleting Network Policy : ingress-on-service
2025-01-06 02:06:48,343 - INFO - Deleting service : ctest-nginx-svc-11022438
2025-01-06 02:06:48,794 - DEBUG - No XMPP flaps were noticed during the test
2025-01-06 02:06:48,794 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:14]
2025-01-06 02:06:48,794 - INFO - --------------------------------------------------------------------------------
2025-01-06 02:06:48,799 - INFO - Deleting pod ctest-new-default-04656451:ctest-pod-28190471
2025-01-06 02:06:48,799 - INFO - Deleting pod ctest-new-default-04656451:ctest-pod-89480523
2025-01-06 02:06:48,800 - INFO - Deleting pod ctest-non-default-63389887:ctest-pod-83273086
2025-01-06 02:06:48,800 - INFO - Deleting pod ctest-non-default-63389887:ctest-pod-33005488
2025-01-06 02:06:48,800 - INFO - Deleting pod ctest-new-default-04656451:ctest-pod-50743923
2025-01-06 02:06:48,801 - INFO - Deleting pod ctest-new-default-04656451:ctest-pod-86431628
2025-01-06 02:06:48,801 - INFO - Deleting pod ctest-non-default-63389887:ctest-pod-29627795
2025-01-06 02:06:48,801 - INFO - Deleting pod ctest-non-default-63389887:ctest-pod-43910232
2025-01-06 02:06:48,888 - WARNING - Pod uuid cdac6568-6098-4916-8c4d-d9991f2ecb06 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,892 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,981 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,982 - WARNING - Pod uuid c14bd24b-7442-47ae-aa32-bba565ee7d4d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,982 - WARNING - Pod uuid 546ea400-226f-4b47-a0d9-2845c485bf10 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,983 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,984 - WARNING - Pod uuid f53c16bd-a4a0-4c25-b513-d721b71cffcb is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:48,984 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:53,899 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:53,900 - DEBUG - Pod cdac6568-6098-4916-8c4d-d9991f2ecb06 is not in agent 10.0.0.34 VM list
2025-01-06 02:06:53,906 - DEBUG - VMI 63e2b830-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:06:53,906 - INFO - Verified that pod ctest-pod-28190471 is removed in agent
2025-01-06 02:06:54,005 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:54,006 - DEBUG - Pod f53c16bd-a4a0-4c25-b513-d721b71cffcb is not in agent 10.0.0.34 VM list
2025-01-06 02:06:54,008 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:54,009 - DEBUG - Pod 546ea400-226f-4b47-a0d9-2845c485bf10 is not in agent 10.0.0.34 VM list
2025-01-06 02:06:54,019 - DEBUG - VMI 64faa9c6-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:06:54,019 - INFO - Verified that pod ctest-pod-83273086 is removed in agent
2025-01-06 02:06:54,020 - DEBUG - VMI 645aac8c-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:06:54,020 - INFO - Verified that pod ctest-pod-89480523 is removed in agent
2025-01-06 02:06:54,041 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:54,041 - DEBUG - Pod c14bd24b-7442-47ae-aa32-bba565ee7d4d is not in agent 10.0.0.34 VM list
2025-01-06 02:06:54,048 - DEBUG - VMI 6577f034-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:06:54,048 - INFO - Verified that pod ctest-pod-33005488 is removed in agent
2025-01-06 02:06:58,908 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:59,018 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:59,018 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:06:59,050 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:03,916 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:04,030 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:04,030 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:04,058 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:08,924 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:09,045 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:09,046 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:09,066 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:13,932 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:14,057 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:14,058 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:14,074 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:18,956 - WARNING - Pod uuid 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:19,070 - WARNING - Pod uuid ca363076-2cdb-4e99-9fb7-bbf9fc49464d is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:19,070 - WARNING - Pod uuid f2b96afc-18b5-415a-b1f3-6de39f914229 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:19,082 - WARNING - Pod uuid 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is still seen in agent 10.0.0.34 VM list
2025-01-06 02:07:23,964 - DEBUG - Pod 8d59364e-3d39-450f-a2e0-ac6e4a709b0b is not in agent 10.0.0.34 VM list
2025-01-06 02:07:23,971 - DEBUG - VMI 6602ffd0-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:07:23,971 - INFO - Verified that pod ctest-pod-50743923 is removed in agent
2025-01-06 02:07:24,082 - DEBUG - Pod ca363076-2cdb-4e99-9fb7-bbf9fc49464d is not in agent 10.0.0.34 VM list
2025-01-06 02:07:24,086 - DEBUG - Pod f2b96afc-18b5-415a-b1f3-6de39f914229 is not in agent 10.0.0.34 VM list
2025-01-06 02:07:24,097 - DEBUG - VMI 676706c8-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:07:24,097 - INFO - Verified that pod ctest-pod-43910232 is removed in agent
2025-01-06 02:07:24,099 - DEBUG - VMI 66ef5416-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:07:24,099 - INFO - Verified that pod ctest-pod-29627795 is removed in agent
2025-01-06 02:07:24,099 - DEBUG - Pod 6213f40a-c69c-45b6-9c83-2b17e0f398b3 is not in agent 10.0.0.34 VM list
2025-01-06 02:07:24,106 - DEBUG - VMI 667c8bd4-cbd2-11ef-b954-fa163e6f847a is removed from agent 10.0.0.34
2025-01-06 02:07:24,106 - INFO - Verified that pod ctest-pod-86431628 is removed in agent
2025-01-06 02:07:24,106 - INFO - Deleting namespace ctest-non-default-63389887
2025-01-06 02:07:24,129 - DEBUG - Namespace ctest-non-default-63389887 still in k8s
2025-01-06 02:07:26,134 - DEBUG - Namespace ctest-non-default-63389887 still in k8s
2025-01-06 02:07:28,140 - DEBUG - Namespace ctest-non-default-63389887 still in k8s
2025-01-06 02:07:30,143 - DEBUG - Namespace ctest-non-default-63389887 is not in k8s
2025-01-06 02:07:30,143 - DEBUG - No need to check project deletion in contrail-api
2025-01-06 02:07:30,143 - INFO - Deleting namespace ctest-new-default-04656451
2025-01-06 02:07:30,151 - DEBUG - Namespace ctest-new-default-04656451 still in k8s
2025-01-06 02:07:32,155 - DEBUG - Namespace ctest-new-default-04656451 still in k8s
2025-01-06 02:07:34,160 - DEBUG - Namespace ctest-new-default-04656451 still in k8s
2025-01-06 02:07:36,163 - DEBUG - Namespace ctest-new-default-04656451 is not in k8s
2025-01-06 02:07:36,164 - DEBUG - No need to check project deletion in contrail-api