2025-08-06 07:30:54,419 - INFO - Using existing project ['default-domain', 'admin'](adb0a682-d2c0-454a-9f6b-407a61f242d6) 2025-08-06 07:30:54,438 - INFO - ================================================================================ 2025-08-06 07:30:54,438 - INFO - STARTING TEST : test_db_nodemgr_status 2025-08-06 07:30:54,439 - INFO - TEST DESCRIPTION : Test to verify db nodemgr status 2025-08-06 07:30:55,126 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:30:55,126 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:30:55,126 - INFO - 2025-08-06 07:30:55,126 - DEBUG - Verifying if db node_mgr running... 2025-08-06 07:30:55,126 - INFO - Verifying through collector 10.0.0.38 for db node cn-jenkins-deploy-platform-ansible-os-3761-1. 2025-08-06 07:30:55,126 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3761-1.?flat 2025-08-06 07:30:55,439 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3761-1.?flat 2025-08-06 07:30:55,533 - INFO - Verifying through collector 10.0.0.38 for db node cn-jenkins-deploy-platform-ansible-os-3761-2. 2025-08-06 07:30:55,533 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3761-2.?flat 2025-08-06 07:30:55,622 - INFO - Verifying through collector 10.0.0.38 for db node cn-jenkins-deploy-platform-ansible-os-3761-3. 2025-08-06 07:30:55,622 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3761-3.?flat 2025-08-06 07:30:56,395 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:30:56,395 - INFO - END TEST : test_db_nodemgr_status : PASSED[0:00:02] 2025-08-06 07:30:56,396 - INFO - -------------------------------------------------------------------------------- 2025-08-06 07:30:56,398 - INFO - ================================================================================ 2025-08-06 07:30:56,398 - INFO - STARTING TEST : test_verify_generator_collector_connections 2025-08-06 07:30:56,398 - INFO - TEST DESCRIPTION : Description: Verify generator:module connections to collector 1.Verify all generators connected to collector - fails otherwise 2.Get the xmpp peers in vrouter uve and get the active xmpp peer out of it 3.Verify from agent introspect that active xmpp matches with step 2 - fails otherwise 4.Get bgp peers from bgp-peer uve and verify from control node introspect that that matches - fails otherwise Maintainer: sandipd@juniper.net 2025-08-06 07:30:57,042 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:30:57,042 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:30:57,042 - INFO - 2025-08-06 07:30:57,049 - INFO - IP 10.0.0.48 port 8083 connected to collector 10.0.0.38 2025-08-06 07:30:57,056 - INFO - IP 10.0.0.254 port 8083 connected to collector 10.0.0.38 2025-08-06 07:30:57,062 - INFO - IP 10.0.0.38 port 8083 connected to collector 10.0.0.38 2025-08-06 07:30:57,067 - INFO - IP 10.0.0.48 port 8084 connected to collector 10.0.0.38 2025-08-06 07:30:57,074 - INFO - IP 10.0.0.254 port 8084 connected to collector 10.0.0.38 2025-08-06 07:30:57,079 - INFO - IP 10.0.0.38 port 8084 connected to collector 10.0.0.254 2025-08-06 07:30:57,091 - INFO - IP 10.0.0.38 port 8087 connected to collector 10.0.0.38 2025-08-06 07:30:57,098 - INFO - IP 10.0.0.254 port 8088 connected to collector 10.0.0.254 2025-08-06 07:30:57,103 - INFO - IP 10.0.0.48 port 8090 connected to collector 10.0.0.48 2025-08-06 07:30:57,108 - INFO - IP 10.0.0.254 port 8090 connected to collector 10.0.0.38 2025-08-06 07:30:57,114 - INFO - IP 10.0.0.38 port 8090 connected to collector 10.0.0.38 2025-08-06 07:30:57,128 - INFO - IP 10.0.0.48 port 8091 connected to collector 10.0.0.48 2025-08-06 07:30:57,137 - INFO - IP 10.0.0.254 port 8091 connected to collector 10.0.0.48 2025-08-06 07:30:57,145 - INFO - IP 10.0.0.38 port 8091 connected to collector 10.0.0.48 2025-08-06 07:30:57,145 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,288 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1:Control:contrail-control:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,289 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,405 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1:Control:contrail-dns:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,406 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,493 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2:Control:contrail-control:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,493 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,531 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2:Control:contrail-dns:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,531 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,605 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3:Control:contrail-control:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,605 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,650 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3:Control:contrail-dns:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,650 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,717 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3761-1:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,717 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,772 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3761-2:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,772 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,802 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Config:contrail-schema:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,802 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,827 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Config:contrail-svc-monitor:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,851 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,886 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Config:contrail-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,892 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,925 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Config:contrail-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,926 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:57,959 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Config:contrail-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:57,982 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,012 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,012 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,045 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,045 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,076 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,076 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,099 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,099 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,121 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,121 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,147 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,147 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,170 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,170 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,193 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,193 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,216 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,216 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,262 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-collector:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,262 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,295 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Database:contrail-query-engine:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,296 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,334 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-collector:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,334 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,379 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Database:contrail-query-engine:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,379 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,430 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-collector:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,431 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:30:58,494 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Database:contrail-query-engine:0 is connected to collector 10.0.0.48 2025-08-06 07:30:58,545 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3761-1?flat 2025-08-06 07:30:58,850 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3761-1?flat 2025-08-06 07:30:58,943 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3761-1 vrouter uve is 10.20.0.14 2025-08-06 07:30:58,969 - DEBUG - Requesting: http://10.20.0.193:8085/Snh_AgentXmppConnectionStatusReq? 2025-08-06 07:30:58,986 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3761-1 agent introspect is 10.20.0.14 2025-08-06 07:30:59,049 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3761-2?flat 2025-08-06 07:30:59,100 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3761-2 vrouter uve is 10.20.0.254 2025-08-06 07:30:59,126 - DEBUG - Requesting: http://10.20.0.129:8085/Snh_AgentXmppConnectionStatusReq? 2025-08-06 07:30:59,157 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3761-2 agent introspect is 10.20.0.254 2025-08-06 07:30:59,209 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3761-1?flat 2025-08-06 07:30:59,317 - INFO - Vrouter an-jenkins-deploy-platform-ansible-os-3761-1 connected to {'10.20.0.25', '10.20.0.14'} xmpp peers out of {'10.20.0.14', '10.20.0.25', '10.20.0.254'} 2025-08-06 07:30:59,317 - INFO - xmpp peer correctly displayed as ['10.20.0.25', '10.20.0.14'] for vrouter an-jenkins-deploy-platform-ansible-os-3761-1 2025-08-06 07:30:59,371 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3761-2?flat 2025-08-06 07:30:59,427 - INFO - Vrouter an-jenkins-deploy-platform-ansible-os-3761-2 connected to {'10.20.0.14', '10.20.0.254'} xmpp peers out of {'10.20.0.14', '10.20.0.25', '10.20.0.254'} 2025-08-06 07:30:59,427 - INFO - xmpp peer correctly displayed as ['10.20.0.14', '10.20.0.254'] for vrouter an-jenkins-deploy-platform-ansible-os-3761-2 2025-08-06 07:30:59,428 - INFO - Verifying through opserver in 10.0.0.38 2025-08-06 07:30:59,428 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3761-1?flat 2025-08-06 07:30:59,464 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3761-2?flat 2025-08-06 07:30:59,497 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3761-3?flat 2025-08-06 07:30:59,528 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3761-1?flat 2025-08-06 07:30:59,558 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3761-2?flat 2025-08-06 07:30:59,588 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3761-3?flat 2025-08-06 07:30:59,613 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3761-1 bgp-router uve 2025-08-06 07:30:59,613 - INFO - Bgp peers = 2 2025-08-06 07:30:59,613 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3761-2 bgp-router uve 2025-08-06 07:30:59,613 - INFO - Bgp peers = 2 2025-08-06 07:30:59,613 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3761-3 bgp-router uve 2025-08-06 07:30:59,613 - INFO - Bgp peers = 2 2025-08-06 07:30:59,613 - INFO - Total xmpp peer connections found = 4 2025-08-06 07:31:00,277 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:31:00,277 - INFO - END TEST : test_verify_generator_collector_connections : PASSED[0:00:04] 2025-08-06 07:31:00,277 - INFO - -------------------------------------------------------------------------------- 2025-08-06 07:31:00,279 - INFO - ================================================================================ 2025-08-06 07:31:00,279 - INFO - STARTING TEST : test_verify_generator_connections_to_collector_node 2025-08-06 07:31:00,280 - INFO - TEST DESCRIPTION : Test to validate generator connections 2025-08-06 07:31:00,944 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:31:00,944 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:31:00,945 - INFO - 2025-08-06 07:31:00,945 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:00,988 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3761-1:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.48 2025-08-06 07:31:00,989 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,013 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3761-1:Compute:contrail-vrouter-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,013 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,067 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3761-2:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,067 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,092 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3761-2:Compute:contrail-vrouter-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,092 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,114 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,114 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,154 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-collector:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,154 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,179 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,179 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,215 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,215 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,257 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-collector:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,257 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,280 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,280 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,309 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,309 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,343 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-collector:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,343 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,373 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,373 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,417 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Database:contrail-query-engine:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,417 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,441 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,441 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,483 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Database:contrail-query-engine:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,483 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,514 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,514 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,556 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Database:contrail-query-engine:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,556 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,594 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,594 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,617 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1.:Config:contrail-api:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,617 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,641 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2.:Config:contrail-api:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,641 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,669 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3.:Config:contrail-api:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,669 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,723 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1:Control:contrail-control:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,723 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,749 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,749 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,799 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-1:Control:contrail-dns:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,799 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,881 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2:Control:contrail-control:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,882 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,924 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,925 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:01,963 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-2:Control:contrail-dns:0 is connected to collector 10.0.0.48 2025-08-06 07:31:01,963 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:02,017 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3:Control:contrail-control:0 is connected to collector 10.0.0.48 2025-08-06 07:31:02,017 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:02,044 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.48 2025-08-06 07:31:02,044 - DEBUG - Verifying through opserver in 10.0.0.48 2025-08-06 07:31:02,089 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3761-3:Control:contrail-dns:0 is connected to collector 10.0.0.48 2025-08-06 07:31:02,720 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:31:02,720 - INFO - END TEST : test_verify_generator_connections_to_collector_node : PASSED[0:00:02] 2025-08-06 07:31:02,721 - INFO - -------------------------------------------------------------------------------- 2025-08-06 07:31:02,723 - INFO - ================================================================================ 2025-08-06 07:31:02,723 - INFO - STARTING TEST : test_verify_process_status_agent 2025-08-06 07:31:02,723 - INFO - TEST DESCRIPTION : Test to validate process_status 2025-08-06 07:31:03,382 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:31:03,383 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:31:03,383 - INFO - 2025-08-06 07:31:03,424 - WARNING - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.20.0.254:5269'] servers NOT UP 2025-08-06 07:31:03,424 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.20.0.14:5269'] servers UP 2025-08-06 07:31:03,424 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1 connected to xmpp 10.20.0.14 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.20.0.25:5269'] servers UP 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1 connected to xmpp 10.20.0.25 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.20.0.254:53'] servers UP 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1 connected to dns 10.20.0.254 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.20.0.14:53'] servers UP 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1 connected to dns 10.20.0.14 2025-08-06 07:31:03,425 - WARNING - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.20.0.25:53'] servers NOT UP 2025-08-06 07:31:03,425 - WARNING - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:03,425 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.0.0.254:8086'] servers UP 2025-08-06 07:31:03,426 - INFO - an-jenkins-deploy-platform-ansible-os-3761-1 connected to collector 10.0.0.254 2025-08-06 07:31:03,426 - WARNING - an-jenkins-deploy-platform-ansible-os-3761-1:contrail-vrouter-agent module connection to ['10.0.0.38:8086'] servers NOT UP 2025-08-06 07:31:04,086 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:31:04,086 - INFO - END TEST : test_verify_process_status_agent : PASSED[0:00:02] 2025-08-06 07:31:04,086 - INFO - -------------------------------------------------------------------------------- 2025-08-06 07:31:04,089 - INFO - ================================================================================ 2025-08-06 07:31:04,089 - INFO - STARTING TEST : test_verify_process_status_analytics_node 2025-08-06 07:31:04,089 - INFO - TEST DESCRIPTION : Test to validate process_status-Analytics-Node 2025-08-06 07:31:04,791 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:31:04,791 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:31:04,791 - INFO - 2025-08-06 07:31:05,175 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-collector module connection to ['10.0.0.48:8086'] servers UP 2025-08-06 07:31:05,175 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-collector module connection to ['10.0.0.48:6379'] servers NOT UP 2025-08-06 07:31:05,175 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-collector module connection to ['10.0.0.254:6379'] servers NOT UP 2025-08-06 07:31:05,175 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP 2025-08-06 07:31:05,188 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-08-06 07:31:05,189 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-collector module connection to ['10.0.0.48:9042'] servers UP 2025-08-06 07:31:05,260 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:05,261 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['10.0.0.254:8086'] servers UP 2025-08-06 07:31:05,261 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['10.0.0.48:6379'] servers NOT UP 2025-08-06 07:31:05,261 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['10.0.0.254:6379'] servers NOT UP 2025-08-06 07:31:05,261 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP 2025-08-06 07:31:05,261 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-08-06 07:31:05,261 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-collector module connection to ['10.0.0.48:9042'] servers UP 2025-08-06 07:31:05,327 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:05,327 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:05,327 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:05,328 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.48:6379'] servers NOT UP 2025-08-06 07:31:05,328 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.254:6379'] servers NOT UP 2025-08-06 07:31:05,328 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP 2025-08-06 07:31:05,328 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-08-06 07:31:05,328 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-collector module connection to ['10.0.0.48:9042'] servers UP 2025-08-06 07:31:05,393 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-analytics-api module connection to ['10.0.0.48:8086'] servers UP 2025-08-06 07:31:05,437 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-analytics-api module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:05,438 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-analytics-api module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:05,438 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-analytics-api module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:05,488 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-analytics-api module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:05,488 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-analytics-api module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:05,489 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-analytics-api module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:05,530 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-analytics-nodemgr module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:05,530 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-analytics-nodemgr module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:05,530 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-analytics-nodemgr module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:05,592 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2.:contrail-analytics-nodemgr module connection to ['10.0.0.48:8086'] servers UP 2025-08-06 07:31:05,647 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3.:contrail-analytics-nodemgr module connection to ['10.0.0.48:8086'] servers UP 2025-08-06 07:31:06,293 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:31:06,294 - INFO - END TEST : test_verify_process_status_analytics_node : PASSED[0:00:02] 2025-08-06 07:31:06,294 - INFO - -------------------------------------------------------------------------------- 2025-08-06 07:31:06,296 - INFO - ================================================================================ 2025-08-06 07:31:06,296 - INFO - STARTING TEST : test_verify_process_status_config 2025-08-06 07:31:06,296 - INFO - TEST DESCRIPTION : Test to validate process_status-Config 2025-08-06 07:31:06,946 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:31:06,946 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:31:06,946 - INFO - 2025-08-06 07:31:06,978 - WARNING - No object found for module contrail-device-manager 2025-08-06 07:31:07,007 - WARNING - No object found for module contrail-device-manager 2025-08-06 07:31:07,069 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.48:8086 servers NOT UP 2025-08-06 07:31:07,070 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.254:8086 servers NOT UP 2025-08-06 07:31:07,070 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.38:8086 servers UP 2025-08-06 07:31:07,070 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.48:2181 servers UP 2025-08-06 07:31:07,071 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.48:9161 servers NOT UP 2025-08-06 07:31:07,071 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.254:9161 servers NOT UP 2025-08-06 07:31:07,072 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to 10.0.0.38:9161 servers NOT UP 2025-08-06 07:31:07,072 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3761-1.*:9161 servers NOT UP 2025-08-06 07:31:07,072 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3761-2.*:9161 servers NOT UP 2025-08-06 07:31:07,073 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3761-3.*:9161 servers NOT UP 2025-08-06 07:31:07,720 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Wed Aug 6 07:31:07 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_process_status_config(self=) 333 334 ''' 335 self.analytics_obj.verify_process_and_connection_infos_config() 336 337 @test.attr(type=['cb_sanity', 'sanity', 'dev_sanity_dpdk']) self = self.analytics_obj = self.analytics_obj.verify_process_and_connection_infos_config = > /contrail-test/tcutils/collector/analytics_tests.py in verify_process_and_connection_infos_config(self=) 4218 'contrail-api',\ 4219 server,node = cfgm) 4220 assert result 4221 result = False 4222 result = False AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 335, in test_verify_process_status_config self.analytics_obj.verify_process_and_connection_infos_config() File "/contrail-test/tcutils/collector/analytics_tests.py", line 4220, in verify_process_and_connection_infos_config assert result AssertionError 2025-08-06 07:31:07,738 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:31:07,738 - INFO - 2025-08-06 07:31:07,738 - INFO - END TEST : test_verify_process_status_config : FAILED[0:00:01] 2025-08-06 07:31:07,739 - INFO - -------------------------------------------------------------------------------- 2025-08-06 07:31:07,745 - INFO - ================================================================================ 2025-08-06 07:31:07,746 - INFO - STARTING TEST : test_verify_process_status_control_node 2025-08-06 07:31:07,746 - INFO - TEST DESCRIPTION : Test to validate process_status-Control-Node 2025-08-06 07:31:08,403 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.25': '0', '10.20.0.14': '0'}, '10.0.0.57': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-08-06 07:31:08,403 - INFO - Initial checks done. Running the testcase now 2025-08-06 07:31:08,403 - INFO - 2025-08-06 07:31:08,429 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1:contrail-control module connection to 10.0.0.48:9041 servers UP 2025-08-06 07:31:08,429 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1:contrail-control module connection to 10.0.0.48:5673 servers UP 2025-08-06 07:31:08,429 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1:contrail-control module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:08,429 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-1:contrail-control module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:08,429 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-1:contrail-control module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:08,461 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2:contrail-control module connection to 10.0.0.48:9041 servers UP 2025-08-06 07:31:08,461 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2:contrail-control module connection to 10.0.0.48:5673 servers UP 2025-08-06 07:31:08,461 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2:contrail-control module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:08,462 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-2:contrail-control module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:08,462 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-2:contrail-control module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:08,496 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3:contrail-control module connection to 10.0.0.48:9041 servers UP 2025-08-06 07:31:08,496 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3:contrail-control module connection to 10.0.0.48:5673 servers UP 2025-08-06 07:31:08,496 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3:contrail-control module connection to ['10.0.0.48:8086'] servers NOT UP 2025-08-06 07:31:08,496 - WARNING - cn-jenkins-deploy-platform-ansible-os-3761-3:contrail-control module connection to ['10.0.0.254:8086'] servers NOT UP 2025-08-06 07:31:08,496 - INFO - cn-jenkins-deploy-platform-ansible-os-3761-3:contrail-control module connection to ['10.0.0.38:8086'] servers UP 2025-08-06 07:31:09,172 - DEBUG - No XMPP flaps were noticed during the test 2025-08-06 07:31:09,172 - INFO - END TEST : test_verify_process_status_control_node : PASSED[0:00:02] 2025-08-06 07:31:09,172 - INFO - --------------------------------------------------------------------------------