2025-07-24 02:32:11,196 - INFO - Using existing project ['default-domain', 'admin'](ace76439-2a78-4fba-a172-8472d66c3498) 2025-07-24 02:32:11,208 - INFO - ================================================================================ 2025-07-24 02:32:11,208 - INFO - STARTING TEST : test_db_nodemgr_status 2025-07-24 02:32:11,209 - INFO - TEST DESCRIPTION : Test to verify db nodemgr status 2025-07-24 02:32:11,856 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:11,856 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:11,856 - INFO - 2025-07-24 02:32:11,856 - DEBUG - Verifying if db node_mgr running... 2025-07-24 02:32:11,856 - INFO - Verifying through collector 10.0.0.41 for db node cn-jenkins-deploy-platform-ansible-os-3712-1. 2025-07-24 02:32:11,856 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3712-1.?flat 2025-07-24 02:32:12,176 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3712-1.?flat 2025-07-24 02:32:12,270 - INFO - Verifying through collector 10.0.0.41 for db node cn-jenkins-deploy-platform-ansible-os-3712-2. 2025-07-24 02:32:12,270 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3712-2.?flat 2025-07-24 02:32:12,362 - INFO - Verifying through collector 10.0.0.41 for db node cn-jenkins-deploy-platform-ansible-os-3712-3. 2025-07-24 02:32:12,362 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3712-3.?flat 2025-07-24 02:32:13,100 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:13,100 - INFO - END TEST : test_db_nodemgr_status : PASSED[0:00:02] 2025-07-24 02:32:13,100 - INFO - -------------------------------------------------------------------------------- 2025-07-24 02:32:13,104 - INFO - ================================================================================ 2025-07-24 02:32:13,104 - INFO - STARTING TEST : test_verify_generator_collector_connections 2025-07-24 02:32:13,104 - INFO - TEST DESCRIPTION : Description: Verify generator:module connections to collector 1.Verify all generators connected to collector - fails otherwise 2.Get the xmpp peers in vrouter uve and get the active xmpp peer out of it 3.Verify from agent introspect that active xmpp matches with step 2 - fails otherwise 4.Get bgp peers from bgp-peer uve and verify from control node introspect that that matches - fails otherwise Maintainer: sandipd@juniper.net 2025-07-24 02:32:13,772 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:13,772 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:13,773 - INFO - 2025-07-24 02:32:13,779 - INFO - IP 10.0.0.19 port 8083 connected to collector 10.0.0.41 2025-07-24 02:32:13,786 - INFO - IP 10.0.0.38 port 8083 connected to collector 10.0.0.41 2025-07-24 02:32:13,794 - INFO - IP 10.0.0.41 port 8083 connected to collector 10.0.0.41 2025-07-24 02:32:13,798 - INFO - IP 10.0.0.19 port 8084 connected to collector 10.0.0.41 2025-07-24 02:32:13,802 - INFO - IP 10.0.0.38 port 8084 connected to collector 10.0.0.38 2025-07-24 02:32:13,807 - INFO - IP 10.0.0.41 port 8084 connected to collector 10.0.0.41 2025-07-24 02:32:13,811 - INFO - IP 10.0.0.19 port 8087 connected to collector 10.0.0.38 2025-07-24 02:32:13,814 - INFO - IP 10.0.0.19 port 8088 connected to collector 10.0.0.38 2025-07-24 02:32:13,818 - INFO - IP 10.0.0.19 port 8090 connected to collector 10.0.0.41 2025-07-24 02:32:13,822 - INFO - IP 10.0.0.38 port 8090 connected to collector 10.0.0.19 2025-07-24 02:32:13,826 - INFO - IP 10.0.0.41 port 8090 connected to collector 10.0.0.19 2025-07-24 02:32:13,833 - INFO - IP 10.0.0.19 port 8091 connected to collector 10.0.0.38 2025-07-24 02:32:13,841 - INFO - IP 10.0.0.38 port 8091 connected to collector 10.0.0.38 2025-07-24 02:32:13,848 - INFO - IP 10.0.0.41 port 8091 connected to collector 10.0.0.38 2025-07-24 02:32:13,848 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:13,960 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1:Control:contrail-control:0 is connected to collector 10.0.0.19 2025-07-24 02:32:13,960 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,043 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1:Control:contrail-dns:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,043 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,091 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2:Control:contrail-control:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,091 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,126 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2:Control:contrail-dns:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,126 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,178 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3:Control:contrail-control:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,178 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,213 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3:Control:contrail-dns:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,214 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,254 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3712-1:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,255 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,295 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3712-2:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,295 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,317 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Config:contrail-schema:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,317 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,338 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Config:contrail-svc-monitor:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,359 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,387 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Config:contrail-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,387 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,421 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Config:contrail-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,421 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,445 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Config:contrail-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,467 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,491 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,491 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,512 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,513 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,535 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,536 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,558 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,559 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,581 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,582 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,610 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,611 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,639 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,640 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,663 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,664 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,686 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,686 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,723 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-collector:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,723 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,753 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Database:contrail-query-engine:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,753 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,789 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-collector:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,789 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,820 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Database:contrail-query-engine:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,821 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,861 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-collector:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,861 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:14,894 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Database:contrail-query-engine:0 is connected to collector 10.0.0.19 2025-07-24 02:32:14,938 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3712-1?flat 2025-07-24 02:32:15,239 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3712-1?flat 2025-07-24 02:32:15,320 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3712-1 vrouter uve is 10.20.0.129 2025-07-24 02:32:15,346 - DEBUG - Requesting: http://10.20.0.18:8085/Snh_AgentXmppConnectionStatusReq? 2025-07-24 02:32:15,358 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3712-1 agent introspect is 10.20.0.129 2025-07-24 02:32:15,396 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3712-2?flat 2025-07-24 02:32:15,437 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3712-2 vrouter uve is 10.20.0.129 2025-07-24 02:32:15,462 - DEBUG - Requesting: http://10.20.0.249:8085/Snh_AgentXmppConnectionStatusReq? 2025-07-24 02:32:15,475 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3712-2 agent introspect is 10.20.0.129 2025-07-24 02:32:15,520 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3712-1?flat 2025-07-24 02:32:15,610 - INFO - Vrouter an-jenkins-deploy-platform-ansible-os-3712-1 connected to {'10.20.0.129', '10.20.0.17'} xmpp peers out of {'10.20.0.129', '10.20.0.25', '10.20.0.17'} 2025-07-24 02:32:15,610 - INFO - xmpp peer correctly displayed as ['10.20.0.17', '10.20.0.129'] for vrouter an-jenkins-deploy-platform-ansible-os-3712-1 2025-07-24 02:32:15,649 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3712-2?flat 2025-07-24 02:32:15,689 - INFO - Vrouter an-jenkins-deploy-platform-ansible-os-3712-2 connected to {'10.20.0.129', '10.20.0.17'} xmpp peers out of {'10.20.0.129', '10.20.0.25', '10.20.0.17'} 2025-07-24 02:32:15,689 - INFO - xmpp peer correctly displayed as ['10.20.0.129', '10.20.0.17'] for vrouter an-jenkins-deploy-platform-ansible-os-3712-2 2025-07-24 02:32:15,689 - INFO - Verifying through opserver in 10.0.0.41 2025-07-24 02:32:15,689 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3712-1?flat 2025-07-24 02:32:15,717 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3712-2?flat 2025-07-24 02:32:15,744 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3712-3?flat 2025-07-24 02:32:15,773 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3712-1?flat 2025-07-24 02:32:15,802 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3712-2?flat 2025-07-24 02:32:15,842 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3712-3?flat 2025-07-24 02:32:15,879 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3712-1 bgp-router uve 2025-07-24 02:32:15,879 - INFO - Bgp peers = 2 2025-07-24 02:32:15,879 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3712-2 bgp-router uve 2025-07-24 02:32:15,879 - INFO - Bgp peers = 2 2025-07-24 02:32:15,879 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3712-3 bgp-router uve 2025-07-24 02:32:15,880 - INFO - Bgp peers = 2 2025-07-24 02:32:15,880 - INFO - Total xmpp peer connections found = 4 2025-07-24 02:32:16,551 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:16,551 - INFO - END TEST : test_verify_generator_collector_connections : PASSED[0:00:03] 2025-07-24 02:32:16,551 - INFO - -------------------------------------------------------------------------------- 2025-07-24 02:32:16,554 - INFO - ================================================================================ 2025-07-24 02:32:16,554 - INFO - STARTING TEST : test_verify_generator_connections_to_collector_node 2025-07-24 02:32:16,554 - INFO - TEST DESCRIPTION : Test to validate generator connections 2025-07-24 02:32:17,225 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:17,225 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:17,225 - INFO - 2025-07-24 02:32:17,225 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,267 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3712-1:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,267 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,290 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3712-1:Compute:contrail-vrouter-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,290 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,331 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3712-2:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,331 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,352 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3712-2:Compute:contrail-vrouter-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,352 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,373 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,373 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,408 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-collector:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,408 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,442 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,442 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,464 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,465 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,499 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-collector:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,499 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,537 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,537 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,567 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,567 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,610 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-collector:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,610 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,641 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,641 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,685 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Database:contrail-query-engine:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,685 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,711 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,712 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,748 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Database:contrail-query-engine:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,748 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,776 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,777 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,813 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Database:contrail-query-engine:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,814 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,838 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,838 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,862 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1.:Config:contrail-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,862 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,887 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2.:Config:contrail-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,887 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,910 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3.:Config:contrail-api:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,912 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,966 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1:Control:contrail-control:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,966 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:17,992 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:17,992 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,032 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-1:Control:contrail-dns:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,034 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,079 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2:Control:contrail-control:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,079 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,102 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,103 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,140 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-2:Control:contrail-dns:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,141 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,204 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3:Control:contrail-control:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,204 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,231 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,231 - DEBUG - Verifying through opserver in 10.0.0.19 2025-07-24 02:32:18,267 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3712-3:Control:contrail-dns:0 is connected to collector 10.0.0.19 2025-07-24 02:32:18,932 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:18,932 - INFO - END TEST : test_verify_generator_connections_to_collector_node : PASSED[0:00:02] 2025-07-24 02:32:18,932 - INFO - -------------------------------------------------------------------------------- 2025-07-24 02:32:18,935 - INFO - ================================================================================ 2025-07-24 02:32:18,935 - INFO - STARTING TEST : test_verify_process_status_agent 2025-07-24 02:32:18,935 - INFO - TEST DESCRIPTION : Test to validate process_status 2025-07-24 02:32:19,589 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:19,589 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:19,589 - INFO - 2025-07-24 02:32:19,633 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.20.0.17:5269'] servers UP 2025-07-24 02:32:19,633 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1 connected to xmpp 10.20.0.17 2025-07-24 02:32:19,633 - WARNING - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.20.0.25:5269'] servers NOT UP 2025-07-24 02:32:19,633 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.20.0.129:5269'] servers UP 2025-07-24 02:32:19,633 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1 connected to xmpp 10.20.0.129 2025-07-24 02:32:19,633 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.20.0.17:53'] servers UP 2025-07-24 02:32:19,634 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1 connected to dns 10.20.0.17 2025-07-24 02:32:19,634 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.20.0.25:53'] servers UP 2025-07-24 02:32:19,634 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1 connected to dns 10.20.0.25 2025-07-24 02:32:19,634 - WARNING - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.20.0.129:53'] servers NOT UP 2025-07-24 02:32:19,634 - WARNING - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:19,634 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.0.0.38:8086'] servers UP 2025-07-24 02:32:19,634 - INFO - an-jenkins-deploy-platform-ansible-os-3712-1 connected to collector 10.0.0.38 2025-07-24 02:32:19,634 - WARNING - an-jenkins-deploy-platform-ansible-os-3712-1:contrail-vrouter-agent module connection to ['10.0.0.41:8086'] servers NOT UP 2025-07-24 02:32:20,289 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:20,289 - INFO - END TEST : test_verify_process_status_agent : PASSED[0:00:02] 2025-07-24 02:32:20,289 - INFO - -------------------------------------------------------------------------------- 2025-07-24 02:32:20,292 - INFO - ================================================================================ 2025-07-24 02:32:20,292 - INFO - STARTING TEST : test_verify_process_status_analytics_node 2025-07-24 02:32:20,292 - INFO - TEST DESCRIPTION : Test to validate process_status-Analytics-Node 2025-07-24 02:32:20,977 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:20,977 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:20,977 - INFO - 2025-07-24 02:32:21,248 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-collector module connection to ['10.0.0.19:8086'] servers UP 2025-07-24 02:32:21,248 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-collector module connection to ['10.0.0.19:6379'] servers NOT UP 2025-07-24 02:32:21,248 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP 2025-07-24 02:32:21,248 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-collector module connection to ['10.0.0.41:6379'] servers NOT UP 2025-07-24 02:32:21,248 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-07-24 02:32:21,248 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-collector module connection to ['10.0.0.19:9042'] servers UP 2025-07-24 02:32:21,317 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:21,317 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['10.0.0.38:8086'] servers UP 2025-07-24 02:32:21,317 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['10.0.0.19:6379'] servers NOT UP 2025-07-24 02:32:21,317 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP 2025-07-24 02:32:21,317 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['10.0.0.41:6379'] servers NOT UP 2025-07-24 02:32:21,318 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-07-24 02:32:21,318 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-collector module connection to ['10.0.0.19:9042'] servers UP 2025-07-24 02:32:21,365 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:21,365 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.38:8086'] servers NOT UP 2025-07-24 02:32:21,366 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.41:8086'] servers UP 2025-07-24 02:32:21,366 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.19:6379'] servers NOT UP 2025-07-24 02:32:21,366 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP 2025-07-24 02:32:21,366 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.41:6379'] servers NOT UP 2025-07-24 02:32:21,366 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-07-24 02:32:21,367 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-collector module connection to ['10.0.0.19:9042'] servers UP 2025-07-24 02:32:21,407 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-analytics-api module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:21,407 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-analytics-api module connection to ['10.0.0.38:8086'] servers NOT UP 2025-07-24 02:32:21,407 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-analytics-api module connection to ['10.0.0.41:8086'] servers UP 2025-07-24 02:32:21,458 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-analytics-api module connection to ['10.0.0.19:8086'] servers UP 2025-07-24 02:32:21,504 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-analytics-api module connection to ['10.0.0.19:8086'] servers UP 2025-07-24 02:32:21,552 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-analytics-nodemgr module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:21,552 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-analytics-nodemgr module connection to ['10.0.0.38:8086'] servers NOT UP 2025-07-24 02:32:21,552 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-analytics-nodemgr module connection to ['10.0.0.41:8086'] servers UP 2025-07-24 02:32:21,599 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-analytics-nodemgr module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:21,599 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2.:contrail-analytics-nodemgr module connection to ['10.0.0.38:8086'] servers UP 2025-07-24 02:32:21,650 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-analytics-nodemgr module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:21,650 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3.:contrail-analytics-nodemgr module connection to ['10.0.0.38:8086'] servers UP 2025-07-24 02:32:22,304 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:22,304 - INFO - END TEST : test_verify_process_status_analytics_node : PASSED[0:00:02] 2025-07-24 02:32:22,304 - INFO - -------------------------------------------------------------------------------- 2025-07-24 02:32:22,307 - INFO - ================================================================================ 2025-07-24 02:32:22,307 - INFO - STARTING TEST : test_verify_process_status_config 2025-07-24 02:32:22,308 - INFO - TEST DESCRIPTION : Test to validate process_status-Config 2025-07-24 02:32:22,971 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:22,971 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:22,971 - INFO - 2025-07-24 02:32:23,006 - WARNING - No object found for module contrail-device-manager 2025-07-24 02:32:23,077 - WARNING - No object found for module contrail-device-manager 2025-07-24 02:32:23,106 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.19:8086 servers NOT UP 2025-07-24 02:32:23,106 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.38:8086 servers NOT UP 2025-07-24 02:32:23,107 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.41:8086 servers UP 2025-07-24 02:32:23,107 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.19:2181 servers UP 2025-07-24 02:32:23,107 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.19:9161 servers NOT UP 2025-07-24 02:32:23,108 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.38:9161 servers NOT UP 2025-07-24 02:32:23,108 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to 10.0.0.41:9161 servers NOT UP 2025-07-24 02:32:23,108 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3712-1.*:9161 servers NOT UP 2025-07-24 02:32:23,109 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3712-2.*:9161 servers NOT UP 2025-07-24 02:32:23,109 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3712-3.*:9161 servers NOT UP 2025-07-24 02:32:23,760 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Thu Jul 24 02:32:23 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_process_status_config(self=) 333 334 ''' 335 self.analytics_obj.verify_process_and_connection_infos_config() 336 337 @test.attr(type=['cb_sanity', 'sanity', 'dev_sanity_dpdk']) self = self.analytics_obj = self.analytics_obj.verify_process_and_connection_infos_config = > /contrail-test/tcutils/collector/analytics_tests.py in verify_process_and_connection_infos_config(self=) 4218 'contrail-api',\ 4219 server,node = cfgm) 4220 assert result 4221 result = False 4222 result = False AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 335, in test_verify_process_status_config self.analytics_obj.verify_process_and_connection_infos_config() File "/contrail-test/tcutils/collector/analytics_tests.py", line 4220, in verify_process_and_connection_infos_config assert result AssertionError 2025-07-24 02:32:23,775 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:23,775 - INFO - 2025-07-24 02:32:23,776 - INFO - END TEST : test_verify_process_status_config : FAILED[0:00:01] 2025-07-24 02:32:23,776 - INFO - -------------------------------------------------------------------------------- 2025-07-24 02:32:23,779 - INFO - ================================================================================ 2025-07-24 02:32:23,779 - INFO - STARTING TEST : test_verify_process_status_control_node 2025-07-24 02:32:23,779 - INFO - TEST DESCRIPTION : Test to validate process_status-Control-Node 2025-07-24 02:32:24,439 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 02:32:24,439 - INFO - Initial checks done. Running the testcase now 2025-07-24 02:32:24,440 - INFO - 2025-07-24 02:32:24,473 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1:contrail-control module connection to 10.0.0.19:9041 servers UP 2025-07-24 02:32:24,473 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1:contrail-control module connection to 10.0.0.19:5673 servers UP 2025-07-24 02:32:24,474 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1:contrail-control module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:24,474 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-1:contrail-control module connection to ['10.0.0.38:8086'] servers NOT UP 2025-07-24 02:32:24,474 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-1:contrail-control module connection to ['10.0.0.41:8086'] servers UP 2025-07-24 02:32:24,517 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2:contrail-control module connection to 10.0.0.19:9041 servers UP 2025-07-24 02:32:24,518 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2:contrail-control module connection to 10.0.0.19:5673 servers UP 2025-07-24 02:32:24,518 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2:contrail-control module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:24,518 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-2:contrail-control module connection to ['10.0.0.38:8086'] servers NOT UP 2025-07-24 02:32:24,518 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-2:contrail-control module connection to ['10.0.0.41:8086'] servers UP 2025-07-24 02:32:24,555 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3:contrail-control module connection to 10.0.0.19:9041 servers UP 2025-07-24 02:32:24,555 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3:contrail-control module connection to 10.0.0.19:5673 servers UP 2025-07-24 02:32:24,555 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3:contrail-control module connection to ['10.0.0.19:8086'] servers NOT UP 2025-07-24 02:32:24,555 - WARNING - cn-jenkins-deploy-platform-ansible-os-3712-3:contrail-control module connection to ['10.0.0.38:8086'] servers NOT UP 2025-07-24 02:32:24,555 - INFO - cn-jenkins-deploy-platform-ansible-os-3712-3:contrail-control module connection to ['10.0.0.41:8086'] servers UP 2025-07-24 02:32:25,227 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 02:32:25,227 - INFO - END TEST : test_verify_process_status_control_node : PASSED[0:00:02] 2025-07-24 02:32:25,228 - INFO - --------------------------------------------------------------------------------