2025-09-01 Mon 02:03:41:425.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:425.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692221425725 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:41:425.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:41:426.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:426.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692221426221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:426.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:426.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:426.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692221426249 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:41:426.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:41:426.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:41:478.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:478.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692221478554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:478.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:479.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:479.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692221479082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:479.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:479.389 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:41:479.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:41:479.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:479.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692221479681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:479.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:709.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:709.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692221708858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:709.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:709.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:709.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692221709181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:709.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:709.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:41:709.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:41:709.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:709.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692221709827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:709.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:422.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:422.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222422777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:422.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:422.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:422.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222422938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:423.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:423.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:42:423.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:42:423.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:423.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222423298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:423.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:423.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:423.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222423302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:423.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:423.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:423.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222423396 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:42:423.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:42:423.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:423.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222423471 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:42:423.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:42:423.507 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:423.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222423486 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:42:423.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:42:424.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:42:424.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:424.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692222424002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:424.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:424.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:424.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692222424057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:42:424.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:42:469.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:469.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692222469224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:469.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:469.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:469.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692222469495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:469.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:469.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:42:469.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:42:469.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:470.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692222469937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:470.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:496.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:496.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692222496833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:496.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:497.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:497.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692222497198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:497.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:497.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:42:497.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:42:497.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:497.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692222497589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:497.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:965.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:965.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222965697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:965.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:966.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:966.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692222966764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34489 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:966.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:967.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:42:967.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:42:967.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:967.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692222967122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:967.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:523.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:523.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692223523144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:523.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:523.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:523.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692223523337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38729 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:523.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:523.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:43:523.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:43:523.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:523.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692223523806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:523.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:762.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:762.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692223761969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:762.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:762.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:762.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692223762185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:762.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:762.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:43:762.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:43:762.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:762.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692223762565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:762.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:639.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:639.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692224639780 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:639.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:639.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:640.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692224639937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46807 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:640.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:640.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:44:640.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:44:640.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:640.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692224640295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:640.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:414.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:414.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692225413917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:414.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:414.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:414.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692225414371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40243 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:414.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:414.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:45:414.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:45:414.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:414.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692225414935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:414.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:567.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:567.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692226567285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:567.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:567.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:567.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692226567446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:567.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:567.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:46:567.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:46:567.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:567.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692226567800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:567.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:567.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:567.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692226567806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:567.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:567.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:567.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692226567870 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:46:567.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:46:567.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:567.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692226567912 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:46:567.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:46:568.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:568.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692226568391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:568.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:568.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:568.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:46:568.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692226568439 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:46:568.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:46:616.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:616.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692226616046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:616.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:616.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:616.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692226616580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:616.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:616.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:46:616.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:46:616.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:617.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692226616974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:617.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:580.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:580.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692227580218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:580.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:580.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:580.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692227580372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:580.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:580.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:581.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:581.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:581.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692227581082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:581.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:722.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:722.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692227722718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:722.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:722.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:722.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692227722913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:722.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:723.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:723.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:723.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:723.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692227723286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:723.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:723.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:723.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692227723293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:723.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:723.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:723.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692227723397 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:47:723.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:723.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:723.495 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692227723467 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:47:723.501 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:724.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:724.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692227724110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:724.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:724.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:724.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692227724158 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:47:724.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:724.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:47:775.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:775.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692227775456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:775.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:775.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:775.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692227775637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:775.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:775.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:775.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:776.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:776.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692227776065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:776.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:528.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:528.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228528449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:528.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:529.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:529.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692228529175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39013 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:529.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:529.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:48:529.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:48:529.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:529.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228529683 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:529.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:529.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:529.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228529690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:529.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:529.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:529.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228529748 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:48:529.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:48:529.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:529.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228529842 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:48:529.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:48:529.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:529.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228529858 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:48:529.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:48:530.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:530.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692228530412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:530.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:530.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:530.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692228530455 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:48:530.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:48:530.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:48:576.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:576.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692228576696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:576.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:577.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:577.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692228577102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:577.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:577.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:48:577.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:48:577.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:577.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692228577488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:577.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:847.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:847.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692228847203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:847.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:847.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:847.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692228847363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:847.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:847.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:48:847.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:48:847.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:847.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692228847795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:847.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:489.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:489.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692229489215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:489.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:489.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:489.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692229489840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:489.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:490.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:49:490.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:49:490.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:490.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692229490586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:490.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:522.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:522.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692229522650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:522.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:523.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:523.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692229523235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:523.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:523.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:49:523.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:49:523.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:523.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692229523616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:523.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:663.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:663.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692229663079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:663.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:663.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:663.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692229663650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46209 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:663.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:663.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:49:664.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:49:664.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692229664106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:664.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:664.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692229664114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:664.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:664.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692229664198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:49:664.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:49:664.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692229664289 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:49:664.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:49:664.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692229664317 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:49:664.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:49:664.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692229664857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:664.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:664.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:664.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692229664915 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:49:664.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:49:664.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:49:717.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:717.276 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692229717071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:717.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:717.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:717.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692229717224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35049 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:717.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:717.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:49:717.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:49:717.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:717.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692229717621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:717.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:662.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:662.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692230662775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:662.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:662.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:663.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692230662935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37091 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:663.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:663.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:50:663.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:50:663.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:663.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692230663396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:663.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:828.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:828.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692230827986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:828.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:828.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:828.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692230828183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45239 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:828.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:828.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:50:828.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:50:828.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:828.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692230828715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:828.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:665.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:665.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692231665706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:665.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:665.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:665.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692231665860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36391 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:665.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:666.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:51:666.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:51:666.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:666.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692231666241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:666.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:751.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:751.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692232751598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:751.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:751.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:751.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692232751773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37555 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:751.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:752.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:52:752.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:52:752.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:753.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692232752930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:753.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:702.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:702.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233702720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:702.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:702.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:702.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233702931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:702.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:703.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:53:703.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:53:703.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:703.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233703358 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:703.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:703.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:703.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233703363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:703.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:703.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:703.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233703432 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:53:703.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:53:703.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:703.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233703471 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:53:703.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:53:703.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:53:704.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:704.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692233703895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:704.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:704.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:704.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692233703930 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:53:704.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:53:754.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:754.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692233754089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:754.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:754.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:754.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692233754679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:754.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:755.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:53:755.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:53:755.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:755.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692233755157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:755.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:681.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:681.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692234680872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:681.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:681.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:681.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692234681049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40705 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:681.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:681.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:681.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:681.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:681.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692234681503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:681.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:681.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:681.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692234681509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:681.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:681.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:681.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692234681587 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:54:681.623 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:681.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:681.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692234681643 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:54:681.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:682.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:54:682.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:682.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692234682089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:682.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:682.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:682.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692234682140 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:54:682.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:728.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:728.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692234728241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:728.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:728.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:728.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692234728746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:728.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:729.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:729.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:729.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:729.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692234729297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:729.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:790.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:790.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692234790287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:790.389 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:790.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:790.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692234790769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:790.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:791.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:791.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:791.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:791.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692234791284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:791.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:266.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:266.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692235266397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:266.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:266.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:266.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692235266846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:266.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:267.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:55:267.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:267.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:267.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692235267282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:267.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:267.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:267.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692235267288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:267.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:267.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:267.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692235267343 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:55:267.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:55:267.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:267.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692235267420 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:55:267.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:55:267.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:267.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692235267436 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:55:267.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:55:267.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:267.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692235267903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:268.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:268.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:268.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692235267936 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:55:268.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:55:268.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:55:319.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:319.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692235319193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:319.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:319.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:319.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692235319721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:319.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:320.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:55:320.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:320.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:320.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692235320173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:320.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:656.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:656.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692235656428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:656.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:657.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:657.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692235656981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34587 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:657.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:657.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:55:657.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:657.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:657.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692235657377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:657.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:391.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:391.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692236391292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:391.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:391.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:391.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692236391778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:391.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:392.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:56:392.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:56:392.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:392.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692236392819 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:392.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:717.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:717.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692236717387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:717.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:717.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:717.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692236717819 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:717.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:718.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:56:718.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:56:718.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:718.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692236718493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:718.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:001.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:001.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692237001078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:001.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:001.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:001.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692237001627 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:001.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:001.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:002.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:002.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:002.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692237002069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:002.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:002.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:002.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692237002084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:002.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:002.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:002.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692237002144 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:57:002.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:57:002.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:002.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692237002589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:57:002.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:57:002.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:002.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692237002610 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:57:002.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:57:003.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:003.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692237003093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:003.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:003.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:003.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692237003126 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:57:003.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:57:003.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:57:052.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:052.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692237052283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:052.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:052.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:052.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692237052810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44371 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:052.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:053.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:053.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:053.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:053.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692237053146 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:053.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:493.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:493.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692237493546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:493.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:494.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:494.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692237494014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:494.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:494.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:494.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:494.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:494.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692237494462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:494.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:989.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:989.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692237989282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:989.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:989.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:989.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692237989765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:989.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:990.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:990.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:990.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:990.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692237990185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:990.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:229.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:229.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692238228961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:229.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:229.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:229.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692238229491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:229.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:229.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:58:229.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:58:229.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:230.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692238229948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:230.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:078.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:078.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692240078318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:078.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:079.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:079.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692240079169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36063 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:079.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:079.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:00:079.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:00:079.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:079.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692240079544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:079.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:858.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:858.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692240858666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:858.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:859.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:859.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692240859122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:859.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:859.495 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:00:859.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:00:859.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:859.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692240859682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:859.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:859.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:859.741 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692240859690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:859.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:859.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:859.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692240859754 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:00:859.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:00:859.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:859.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692240859808 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:00:859.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:00:860.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:860.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692240860559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:860.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:860.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:860.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:00:860.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692240860611 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:00:860.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:00:909.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:909.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692240909717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:909.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:910.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:910.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692240910400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:910.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:910.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:00:910.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:00:910.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:910.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692240910835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:910.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:454.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:454.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692241454620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:454.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:454.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:454.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692241454818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46405 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:454.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:455.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:455.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:455.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:455.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:01:455.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692241455277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:456.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:456.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:456.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692241455283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:456.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:456.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:456.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692241455346 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:456.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:456.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:456.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692241455392 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:01:456.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:456.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:456.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692241455875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:456.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:456.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:456.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692241455919 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:456.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:507.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:507.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692241507113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:507.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:507.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:507.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692241507291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44763 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:507.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:507.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:507.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:507.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:507.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692241507761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:507.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:928.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:928.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692241927986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:928.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:928.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:928.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692241928166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:928.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:928.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:928.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:928.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:928.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692241928599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:928.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:370.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:370.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692242370087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:370.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:370.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:370.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692242370716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:370.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:371.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:02:371.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:02:371.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:371.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692242371504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:371.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:371.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:371.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692242371510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:371.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:371.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:371.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692242371569 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:02:371.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:02:371.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:371.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692242371644 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:02:371.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:02:371.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:371.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692242371661 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:02:371.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:02:372.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:372.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692242372163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:372.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:372.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:372.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692242372197 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:02:372.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:02:372.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:02:407.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:408.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692242407913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:408.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:408.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:408.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692242408339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34827 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:408.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:408.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:02:408.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:02:408.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:409.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692242408979 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:409.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:418.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:418.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692242418481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:418.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:418.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:418.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692242418628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:418.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:419.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:02:419.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:02:419.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:419.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692242419107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:419.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:319.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:319.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692243319172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:319.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:319.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:319.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692243319592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:319.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:319.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:03:320.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:03:320.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:320.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692243320168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:320.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:931.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:931.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692243931663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:931.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:932.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:932.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692243932635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:932.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:932.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:03:933.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:03:933.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692243933065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:933.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:933.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692243933071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:933.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:933.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692243933121 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:03:933.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:03:933.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692243933196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:03:933.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:03:933.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692243933213 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:03:933.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:03:933.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692243933825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:933.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:933.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:933.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692243933898 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:03:933.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:03:934.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:03:980.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:980.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692243980726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:980.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:981.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:981.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692243981402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39287 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:981.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:981.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:03:981.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:03:981.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:981.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692243981740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:981.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:983.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:983.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692243983180 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:983.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:983.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:983.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692243983903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:983.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:984.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:03:984.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:03:984.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:984.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692243984293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:984.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:04:479.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:04:479.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692244479159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:04:479.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:04:479.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:04:479.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692244479312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:04:479.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:04:479.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:04:479.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:04:479.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:04:479.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692244479749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:04:479.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:010.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:010.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692245010451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:010.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:011.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:011.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692245010979 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:011.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:011.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:05:011.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:05:011.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:011.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692245011350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:011.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:138.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:138.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692245138300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:138.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:138.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:138.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692245138861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:138.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:139.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:05:139.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:05:139.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:139.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692245139251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:139.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:847.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:847.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692246847491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:847.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:848.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:848.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692246848122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45755 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:848.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:848.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:06:848.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:06:848.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:848.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692246848554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:848.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:661.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:661.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692247661501 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 1880 dequeues = 1880 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 1911 dequeues = 1911 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692218682271 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692246848562 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:04:07:661.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:07:661.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:661.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692247661627 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 375 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 426 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 100 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1440 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 411 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 940, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1880, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:07:661.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:07:667.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:668.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247667887 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:04:07:668.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:04:07:668.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:668.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247667922 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 2234544864 used = 32524 free = 0 buffers = 0 cached = 2303930040 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.135 five_min_avg = 0.21 fifteen_min_avg = 0.165 ] cpu_share = 1.525 ] ] 2025-09-01 Mon 02:04:07:668.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:07:668.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:668.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247667927 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.525 used_sys_mem = 1991069780 one_min_cpuload = 0.135 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:04:07:668.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:04:07:668.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:668.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247667951 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 567 in_bytes = 59057 out_tpkts = 617 out_bytes = 34394 exception_packets = 30 exception_packets_allowed = 19 raw_vhost_stats= [ name = vhost0 in_pkts = 567 in_bytes = 59057 out_pkts = 617 out_bytes = 34394 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 528 in_bytes = 30656 out_pkts = 569 out_bytes = 59109 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:07:668.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:07:668.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:668.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247668233 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:07:668.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:07:941.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:941.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692247941868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:941.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:942.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:942.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692247942491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43315 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:942.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:942.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:07:942.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:07:942.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:942.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692247942842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:942.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:942.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:942.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692247942850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:942.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:942.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:942.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692247942927 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:07:942.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:07:942.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:942.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692247942970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:07:943.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:07:943.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:943.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247943573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:943.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:943.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:943.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692247943605 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:07:943.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:07:943.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:07:991.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:991.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692247991844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:991.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:992.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:992.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692247991998 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:992.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:992.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:07:992.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:07:992.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:992.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692247992359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:992.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:682.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:08:682.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:04:08:682.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:08:682.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692248682659 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:682.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:682.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:08:683.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692248682895 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 378 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 428 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 101 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 413 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 945, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1898, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:08:683.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:08:683.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:08:683.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:08:683.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:08:683.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:08:683.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:684.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Idle 2025-09-01 Mon 02:04:08:684.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:684.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692248683922 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 379 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 428 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 101 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 413 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 945, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1900, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:08:684.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:08:684.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:684.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692248683959 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:684.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:684.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:08:684.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:684.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692248684132 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 379 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 429 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 101 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 413 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 946, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1900, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:08:684.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:08:691.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:691.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692248691826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:691.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:692.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:692.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692248692119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44013 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:692.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:692.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:692.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:692.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:692.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692248692544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:692.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:692.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:692.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692248692548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:692.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:692.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:692.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692248692589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:692.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:692.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:692.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692248692656 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:08:692.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:693.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:693.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692248693111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:693.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:693.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:693.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692248693150 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:693.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:693.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:08:746.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:746.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692248746428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:746.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:747.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:747.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692248747029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40147 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:747.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:747.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:747.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:747.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:747.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692248747395 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:747.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:847.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:847.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692248847428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:847.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:847.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:847.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692248847644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44723 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:847.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:847.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:848.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:848.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:848.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:848.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:848.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:848.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:848.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:848.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:848.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848149 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:848.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:848.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:848.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848247 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:848.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:848.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:848.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848261 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:08:848.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:848.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:848.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:848.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:848.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:849.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248848765 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:849.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:848.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:08:899.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:899.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692248898934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:899.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:899.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:899.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692248899107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45927 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:899.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:899.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:899.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:899.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:899.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692248899492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:899.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:901.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:901.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692248901450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:901.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:901.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:901.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692248901544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40563 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:901.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:901.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:901.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:901.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:901.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692248901888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:901.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:656.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:656.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692249656562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:656.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:656.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:656.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692249656741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39483 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:656.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:657.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:09:657.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:09:657.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:657.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692249657122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:657.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:808.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:808.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692249808622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:808.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:809.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:809.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692249809148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:809.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:809.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:09:809.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:09:809.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:809.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692249809566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:809.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:448.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:448.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250448695 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:448.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:448.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:448.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250448892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:448.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:449.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:10:449.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:10:449.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:449.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250449341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:449.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:449.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:449.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250449346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:449.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:449.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:449.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692250449420 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:10:449.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:10:449.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:449.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250449503 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:10:449.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:10:449.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:449.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250449518 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:10:449.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:10:450.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:450.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:10:450.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692250449962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:450.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:450.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:450.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692250449996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:10:450.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:10:502.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:502.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692250502149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:502.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:502.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:502.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692250502310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:502.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:502.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:10:502.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:10:502.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:502.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250502677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:502.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:918.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:918.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692250918010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:918.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:919.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:919.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692250919206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:919.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:919.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:10:919.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:10:919.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:919.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692250919625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:919.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:519.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:520.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692251519837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:520.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:520.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:520.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692251520374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46301 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:520.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:520.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:11:520.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:11:520.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:520.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692251520778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:520.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:735.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:735.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692251735671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:735.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:736.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:736.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692251736110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45353 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:736.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:736.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:11:736.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:11:736.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:736.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692251736455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:736.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:745.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:745.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692251745195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:745.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:745.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:745.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692251745688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42099 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:745.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:746.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:11:746.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:11:746.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:746.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692251746288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:746.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:537.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:537.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692253536901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:537.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:537.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:537.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692253537517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:537.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:537.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:13:537.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:13:537.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:537.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692253537883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:537.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:207.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:53880 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:14:218.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:53882 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:14:951.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:951.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692254951751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:951.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:951.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:951.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254951899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:951.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:952.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:14:952.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:14:952.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:952.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254952310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:952.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:952.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:952.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254952314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:952.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:952.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:953.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254952381 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:14:953.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:953.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:953.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254952420 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:14:953.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:953.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:953.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254952890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:953.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:953.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:953.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692254952915 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:14:953.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:953.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:14:998.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:998.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692254998310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:998.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:998.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:998.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692254998698 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40205 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:998.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:999.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:14:999.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:14:999.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:999.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692254999181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:999.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:660.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:660.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692255660628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:660.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:660.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:660.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692255660805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34143 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:660.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:661.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:661.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:661.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:661.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255661190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:661.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:661.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:661.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255661195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:661.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:661.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:661.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255661249 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:661.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:661.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:661.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692255661478 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:661.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:661.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:661.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692255661493 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:15:661.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:662.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:662.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692255661985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:662.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:662.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:662.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692255662030 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:662.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:662.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:15:707.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:707.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692255707277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:707.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:707.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:707.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692255707846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:707.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:708.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:708.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:708.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:708.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692255708329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:708.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:742.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:742.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692255742361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:742.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:742.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:742.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692255742802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38685 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:742.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:743.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:743.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:743.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:743.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255743242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:743.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:743.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:743.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255743246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:743.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:743.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:743.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255743307 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:743.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:743.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:743.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692255743351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:15:743.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:743.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:743.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692255743796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:743.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:743.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:743.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692255743829 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:743.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:743.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:15:792.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:792.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692255792032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:792.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:792.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:792.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692255792631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:792.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:792.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:793.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:793.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:793.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692255793070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:793.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:034.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:034.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692256034265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:034.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:034.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:034.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692256034650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:034.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:034.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:16:035.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:16:035.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:035.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692256035129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:035.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:720.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:720.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692256720228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:720.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:720.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:720.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692256720387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:720.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:720.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:16:720.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:16:720.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:720.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692256720784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:720.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:779.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:779.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692256779513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:779.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:779.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:779.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692256779703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36455 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:779.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:780.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:16:780.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:16:780.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:780.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692256780242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:780.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:210.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:210.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692257210302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:210.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:210.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:210.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692257210463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32833 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:210.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:210.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:17:210.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:17:210.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:210.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692257210864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:210.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:210.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:210.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692257210869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:210.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:210.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:210.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692257210917 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:17:210.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:17:210.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:211.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692257210989 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:17:211.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:17:211.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:211.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692257211011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:17:211.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:17:211.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:211.623 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692257211541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:211.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:211.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:211.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692257211579 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:17:211.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:17:211.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:17:258.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:258.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692257258724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:258.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:259.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:259.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692257259168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40615 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:259.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:259.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:17:259.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:17:259.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:259.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692257259613 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:259.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:159.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:159.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692258159269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:159.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:160.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:160.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692258160142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:160.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:160.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:18:160.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:18:160.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:160.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692258160587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:160.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:222.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:222.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692258222753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:222.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:223.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:223.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692258223684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:223.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:223.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:18:224.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:18:224.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:224.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692258224042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:224.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:574.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:575.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692258574901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:575.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:575.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:575.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692258575731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33191 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:575.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:576.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:18:576.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:18:576.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:576.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692258576130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:576.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:886.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:886.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692258886347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:886.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:887.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:887.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692258887136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:887.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:887.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:18:887.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:18:887.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:887.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692258887563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:887.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:366.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:366.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692260366194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:366.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:366.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:366.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692260366748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32955 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:366.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:367.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:20:367.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:20:367.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:367.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692260367156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:367.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:012.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:012.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692262012753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:012.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:012.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:012.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692262012918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35633 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:012.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:013.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:013.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:013.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:014.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262013374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:014.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:014.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:014.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262013378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:014.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:014.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:014.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262013430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:014.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:014.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:014.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262013491 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:22:014.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:014.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:014.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262013974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:014.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:014.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:014.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262014029 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:014.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:014.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:22:059.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:059.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692262059173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:059.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:059.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:059.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692262059355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:059.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:059.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:059.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:059.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:059.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692262059812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:059.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:559.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:560.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692262559740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:560.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:560.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:560.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692262560006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:560.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:561.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:561.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:562.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:562.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262562062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:562.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:562.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:562.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262562079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:562.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:562.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:562.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262562179 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:562.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:562.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:562.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262562596 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:562.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:562.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:562.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262562640 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:22:562.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:563.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:563.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692262563345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:563.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:563.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:563.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692262563406 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:563.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:563.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:22:615.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:615.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692262615584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:615.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:616.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:616.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692262616646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35487 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:616.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:616.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:617.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:617.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:617.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262617142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:617.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:824.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:824.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692262824263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:824.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:824.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:824.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262824737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:824.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:825.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:825.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:825.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:825.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262825161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:825.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:825.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:825.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262825166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:825.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:825.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:825.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262825211 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:825.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:825.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:825.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262825252 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:22:825.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:825.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:825.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262825732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:825.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:825.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:825.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262825782 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:22:825.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:22:826.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:22:879.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:879.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692262879186 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:879.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:879.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:879.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692262879656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41739 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:879.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:880.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:880.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:880.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:880.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692262880658 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:880.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:031.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:032.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692263031949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:032.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:032.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:032.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692263032560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38651 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:032.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:032.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:23:032.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:23:033.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:033.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692263032993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:033.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:535.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:535.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692263535266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:535.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:535.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:535.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692263535720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:535.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:535.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:23:536.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:23:536.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:536.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692263536119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:536.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:862.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:862.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692263861897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:862.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:862.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:862.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692263862386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46871 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:862.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:862.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:23:862.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:23:862.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:862.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692263862832 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:862.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:967.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:967.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692263967304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:967.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:968.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:968.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692263968006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:968.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:968.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:23:968.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:23:968.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:968.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692263968434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:968.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:968.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:968.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692263968442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:968.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:968.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:968.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692263968597 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:23:968.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:23:968.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:968.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692263968938 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:23:969.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:23:969.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:969.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692263968964 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:23:969.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:23:969.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:969.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692263969465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:23:969.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:23:969.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:23:969.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692263969501 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:23:969.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:23:969.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:24:016.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:016.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692264016753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:016.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:017.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:017.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692264017149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:017.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:017.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:24:017.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:24:017.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:017.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692264017593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:017.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:926.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:926.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692264926730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:926.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:927.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:927.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692264927198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42235 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:927.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:927.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:24:927.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:24:927.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:927.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692264927677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:927.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:175.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:175.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692265175138 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:175.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:175.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:175.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692265175632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33455 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:175.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:175.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:176.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:176.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:176.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692265176075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:176.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:552.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:552.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692265552233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:552.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:552.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:552.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692265552814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:552.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:553.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:553.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:553.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:553.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692265553263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:553.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:717.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:717.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692265716940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:717.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:717.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:717.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692265717445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40759 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:717.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:717.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:717.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:717.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:717.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692265717891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:717.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:925.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:925.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692266925811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:925.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:926.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:926.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692266926333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45919 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:926.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:926.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:26:926.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:26:926.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:926.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692266926873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:926.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:884.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:884.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692268884321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:884.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:884.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:884.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692268884938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:884.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:885.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:28:885.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:28:885.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:885.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692268885339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:885.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:885.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:885.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692268885345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:885.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:885.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:885.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692268885396 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:28:885.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:28:885.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:885.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692268885439 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:28:885.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:28:886.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:886.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692268886102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:886.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:886.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:886.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692268886143 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:28:886.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:28:886.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:28:938.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:938.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692268938396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:938.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:938.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:938.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692268938659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40733 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:938.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:938.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:28:939.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:28:939.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:939.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692268939117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:939.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:462.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:462.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692269462027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:462.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:462.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:462.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269462612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:462.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:462.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:463.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:463.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:463.097 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692269463069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:463.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:463.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:463.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692269463079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:463.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:463.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:463.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692269463137 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:463.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:463.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:463.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692269463202 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:29:463.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:463.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:463.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269463659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:463.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:463.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:463.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269463696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:463.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:463.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:29:508.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:509.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692269508925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:509.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:509.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:509.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692269509425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36621 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:509.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:509.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:510.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:510.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:510.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692269510098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:510.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:513.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:513.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692269513337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:513.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:513.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:513.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692269513625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:513.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:513.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:513.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:514.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:514.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692269513975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:514.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:514.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:514.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692269513991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:514.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:514.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:514.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692269514048 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:514.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:514.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:514.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692269514524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:514.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:514.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:514.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692269514542 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:29:514.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:515.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:515.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269515199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:515.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:515.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:515.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269515234 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:515.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:515.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:29:560.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:560.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692269560426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:560.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:560.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:560.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692269560626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:560.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:560.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:560.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:561.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:561.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269561008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:561.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:866.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:866.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269866273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:866.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:866.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:866.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692269866431 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:866.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:866.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:866.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:866.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:866.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692269866778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:866.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:455.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:455.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692270455252 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:455.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:455.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:455.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692270455408 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37955 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:455.507 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:455.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:30:455.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:30:455.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:455.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692270455835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:455.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:551.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:551.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692270551226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:551.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:551.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:551.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692270551646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:551.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:551.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:30:552.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:30:552.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:552.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692270552116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:552.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:815.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:815.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692270815021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:815.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:815.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:815.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692270815195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:815.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:815.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:30:815.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:30:815.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:815.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270815811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:815.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:815.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:815.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270815821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:815.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:815.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:815.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270815882 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:30:815.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:30:815.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:815.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270815966 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:30:815.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:30:816.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:816.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270815982 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:30:816.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:30:816.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:816.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692270816477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:816.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:816.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:816.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692270816548 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:30:816.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:30:816.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:30:862.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:862.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270862795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:862.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:863.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:863.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692270863411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37037 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:863.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:863.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:30:863.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:30:863.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:864.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692270863970 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:864.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:872.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:872.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692271872175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:31:872.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:872.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:872.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692271872380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:31:872.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:872.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:31:872.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:31:872.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:872.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692271872789 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:31:872.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:009.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:009.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692272008959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:009.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:009.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:009.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692272009171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:009.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:009.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:009.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:009.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:009.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692272009659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:009.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:310.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:310.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692272310004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:310.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:310.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:310.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692272310166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40825 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:310.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:310.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:310.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:310.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:310.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692272310557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:310.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:640.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:640.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692272640244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:640.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:640.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:640.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692272640756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:640.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:641.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:641.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:641.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:641.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692272641179 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:641.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:726.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:727.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692273726893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:727.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:727.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:727.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692273727823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37293 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:727.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:728.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:33:728.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:33:728.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:728.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692273728176 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:728.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:149.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:149.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692276149798 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:149.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:150.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:150.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276150329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:150.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:150.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:150.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:151.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:36:151.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:151.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276150697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:151.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:151.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:151.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276150710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:151.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:151.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:151.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276150766 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:151.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:151.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:151.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276150810 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:36:151.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:151.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:151.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276151300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:151.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:151.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:151.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276151338 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:151.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:204.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:204.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276204453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:204.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:204.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:204.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276204632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44889 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:204.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:204.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:205.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:205.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:205.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692276205053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:205.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:241.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:241.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276241368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:241.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:241.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:241.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276241545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44259 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:241.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:241.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:241.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:241.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:241.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692276241920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:242.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:242.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:242.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692276241926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:242.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:242.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:242.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692276241982 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:242.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:242.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:242.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692276242104 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:242.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:242.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:242.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692276242119 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:36:242.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:242.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:242.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692276242644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:242.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:242.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:242.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692276242688 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:242.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:242.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:36:292.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:292.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692276292034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:292.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:292.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:292.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276292627 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:292.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:292.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:293.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:293.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:293.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692276293080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:293.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:558.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:558.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276558670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:558.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:558.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:558.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276558834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:558.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:559.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:559.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:559.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:559.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276559206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:559.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:559.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:559.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276559214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:559.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:559.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:559.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276559268 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:559.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:559.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:559.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276559314 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:36:559.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:559.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:559.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:36:559.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276559772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:560.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:560.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:560.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276559822 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:36:560.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:36:610.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:610.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692276610000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:610.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:610.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:610.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692276610177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:610.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:36:610.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:36:610.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:36:610.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:36:610.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692276610603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:36:610.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:186.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:186.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692277186225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:186.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:186.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:186.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692277186385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:186.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:186.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:37:186.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:37:186.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:186.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692277186797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:186.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:346.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:346.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692277346221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:346.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:346.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:346.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692277346375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:346.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:346.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:37:346.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:37:346.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:346.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692277346801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:346.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:661.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:661.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692277661713 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 2212 dequeues = 2212 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 2246 dequeues = 2246 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692248684030 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692277346806 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:04:37:661.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:37:661.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:662.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692277661807 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 443 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 501 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 119 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1694 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 483 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2212, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:37:662.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:37:663.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:663.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692277663671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:663.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:663.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:663.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692277663806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38135 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:663.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:664.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:37:664.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:37:664.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:664.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692277664172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:664.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:668.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:668.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692277668122 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:04:37:668.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:04:37:668.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:668.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692277668164 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 670 in_bytes = 76335 out_tpkts = 727 out_bytes = 40736 exception_packets = 32 exception_packets_allowed = 21 raw_vhost_stats= [ name = vhost0 in_pkts = 670 in_bytes = 76335 out_pkts = 727 out_bytes = 40736 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] total_in_bandwidth_utilization = 7.49184e-15 total_out_bandwidth_utilization = 1.65937e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1382, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3061, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 623 in_bytes = 36368 out_pkts = 672 out_bytes = 76387 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:37:668.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:37:668.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:668.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692277668405 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:37:668.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:37:976.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:976.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692277976324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:976.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:976.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:976.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692277976919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:977.007 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:977.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:37:977.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:37:977.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:977.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692277977332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:977.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:977.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:977.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692277977342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:977.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:977.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:977.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692277977424 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:37:977.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:37:977.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:977.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692277977511 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:37:977.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:37:977.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:977.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692277977534 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:37:977.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:37:978.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:978.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692277977994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:37:978.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:978.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:37:978.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:37:978.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692277978040 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:37:978.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:38:027.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:027.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692278027193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:027.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:027.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:027.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692278027356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41571 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:027.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:027.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:38:027.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:38:027.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:027.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692278027767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:027.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:684.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:38:684.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:04:38:684.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:38:684.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:38:684.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:38:684.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692278684230 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:38:684.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:38:684.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:38:684.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692278684460 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 447 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 504 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1705 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 486 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1113, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2232, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:38:684.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:38:684.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:38:684.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:38:685.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:38:685.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Idle 2025-09-01 Mon 02:04:38:685.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:685.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692278685006 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 504 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1705 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 486 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1113, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2234, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:38:685.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:38:685.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:685.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692278685053 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:38:685.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:38:685.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:38:685.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:685.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692278685134 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 505 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1705 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 486 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1114, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2234, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:38:685.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:38:954.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:954.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692278954895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:955.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:955.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:955.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692278955515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45489 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:955.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:955.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:38:955.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:38:955.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:955.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692278955873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:955.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:149.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:149.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692279148928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:149.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:149.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:149.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692279149379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:149.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:149.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:149.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:149.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:149.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692279149909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:149.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:218.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:219.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692279218933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:219.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:219.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:219.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692279219515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40797 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:219.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:219.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:219.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:219.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:219.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692279219949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:219.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:482.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:482.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692279482306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:482.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:482.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:482.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692279482847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:482.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:39:483.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:39:483.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:39:483.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:39:483.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692279483235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:39:483.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:990.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:990.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692280990060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:990.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:991.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:991.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692280991154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44405 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:991.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:991.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:40:991.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:40:991.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:40:991.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692280991551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:40:991.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:191.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:191.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283191386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:191.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:191.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:191.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692283191906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:191.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:192.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:192.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:192.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:192.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283192427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:192.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:192.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:192.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283192432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:192.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:192.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:192.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283192487 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:192.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:192.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:192.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283192563 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:43:192.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:193.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:193.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692283193071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:193.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:193.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:193.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692283193108 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:193.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:193.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:43:246.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:246.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283246500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:246.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:247.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:247.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692283247099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:247.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:247.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:247.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:247.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:247.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692283247699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:247.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:290.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:290.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283290059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:290.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:290.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:290.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283290626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:290.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:290.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:291.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:291.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:43:291.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:291.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283291102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:291.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:292.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:292.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283291279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:292.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:292.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:292.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283291348 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:292.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:292.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:292.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283291388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:43:292.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:292.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:292.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283291825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:292.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:292.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:292.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283291847 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:292.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:337.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:337.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692283336996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:337.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:337.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:337.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692283337535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:337.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:337.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:337.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:337.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:337.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692283337878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:337.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:396.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:396.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692283396093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:396.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:396.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:396.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692283396690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:396.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:396.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:397.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:397.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:397.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692283397135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:397.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:397.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:397.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283397193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:397.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:397.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:397.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692283397260 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:397.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:398.021 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:398.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692283397440 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:398.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:398.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:398.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692283397456 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:43:398.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:398.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:398.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692283397941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:398.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:398.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:398.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692283397976 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:43:398.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:43:398.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:43:446.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:446.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692283446367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:446.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:446.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:446.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692283446782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:446.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:43:447.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:43:447.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:43:447.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:447.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692283447209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:43:447.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:228.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:228.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692284228834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:229.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:229.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:229.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692284229349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34101 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:229.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:229.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:44:229.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:44:229.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:229.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692284229794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:229.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:409.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:409.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692284409023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:409.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:409.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:409.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692284409531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:409.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:409.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:44:409.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:44:409.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:410.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692284409926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:410.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:518.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:518.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692284518321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:518.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:518.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:518.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692284518786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37577 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:518.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:44:519.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:44:519.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:44:519.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:519.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692284519166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:44:519.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:167.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:167.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692285167656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:167.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:168.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:168.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692285168117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41169 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:168.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:168.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:45:168.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:45:168.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:168.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692285168580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:168.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:168.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:168.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692285168585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:168.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:168.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:168.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692285168639 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:45:168.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:45:168.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:168.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692285168729 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:45:168.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:45:168.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:168.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692285168755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:45:168.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:45:169.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:169.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692285169297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:169.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:169.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:169.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692285169333 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:45:169.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:45:169.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:45:222.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:222.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692285222492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:222.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:222.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:222.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692285222918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42865 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:222.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:45:223.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:45:223.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:45:223.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:45:223.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692285223373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:45:223.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:141.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:141.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692286141524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:141.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:142.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:142.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692286142088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40613 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:142.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:142.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:46:142.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:46:142.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:142.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692286142585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:142.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:173.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:174.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692286173927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:174.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:174.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:174.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692286174096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:174.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:174.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:46:174.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:46:174.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:174.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692286174491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:174.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:355.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:355.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692286355303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:355.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:356.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:356.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692286356114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:356.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:356.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:46:356.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:46:356.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:356.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692286356533 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:356.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:408.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:408.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692286408029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:408.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:408.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:408.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692286408848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:408.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:46:409.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:46:409.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:46:409.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:46:409.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692286409208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:46:409.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:48:194.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:194.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692288194768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:195.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:48:195.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:195.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692288194929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35593 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:195.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:48:195.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:48:195.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:48:195.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:195.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692288195418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:195.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:918.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:918.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692289918581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:918.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:919.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:919.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692289919131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39615 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:919.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:919.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:919.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:919.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:920.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:49:920.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289919477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:920.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:920.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:920.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289919483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:920.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:920.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:920.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289919537 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:920.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:920.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:920.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289919579 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:49:920.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:920.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:920.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289920038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:920.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:920.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:920.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289920069 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:920.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:956.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:956.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692289956665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:956.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:957.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:957.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692289957221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35875 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:957.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:957.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:957.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:957.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:957.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692289957680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:957.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:957.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:957.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692289957690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:957.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:957.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:957.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692289957740 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:957.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:958.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:958.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692289957948 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:958.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:958.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:958.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692289957969 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:49:958.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:958.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:958.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692289958424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:958.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:958.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:958.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692289958448 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:49:958.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:49:958.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:49:973.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:973.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692289973135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:973.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:973.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:973.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692289973574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:973.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:49:973.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:49:973.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:49:974.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:49:974.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692289974030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:49:974.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:007.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:007.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692290007592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:007.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:008.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:008.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692290008161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:008.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:008.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:008.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:008.595 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:008.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692290008585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:008.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:045.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:045.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692290045321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:045.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:045.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:045.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692290045882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:045.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:046.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:046.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:046.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:046.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692290046343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:046.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:046.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:046.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692290046352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:046.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:046.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:046.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692290046443 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:50:046.858 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:50:046.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:046.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692290046512 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:50:046.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:50:047.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:047.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692290046963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:047.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:047.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:047.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692290046993 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:50:047.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:50:047.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:50:098.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:098.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692290098326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:098.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:099.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:099.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692290098989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34025 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:099.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:099.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:099.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:099.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:099.389 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692290099335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:099.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:935.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:936.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692290935779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:936.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:50:936.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:936.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:936.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:50:936.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692290935936 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:936.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:936.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:50:936.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692290936427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:50:936.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:018.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:018.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692291018120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:018.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:018.741 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:018.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692291018723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:018.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:018.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:51:019.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:51:019.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:019.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692291019470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:019.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:071.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:071.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692291071400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:071.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:071.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:071.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692291071869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:071.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:072.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:51:072.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:51:072.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:072.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692291072457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:072.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:975.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:975.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692291975506 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:975.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:976.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:976.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692291976384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38775 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:976.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:976.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:51:976.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:51:976.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:976.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291976753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:976.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:976.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:976.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291976757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:976.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:977.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:977.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291976887 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:51:977.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:51:977.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:977.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291976994 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:51:977.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:51:977.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:977.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291977008 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:51:977.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:51:977.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:977.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291977534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:51:977.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:51:977.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:51:977.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692291977562 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:51:977.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:51:977.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:52:030.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:031.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692292030876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:031.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:031.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:031.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692292031638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:031.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:031.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:032.021 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:032.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:032.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692292032047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:032.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:736.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:736.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692292736593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:736.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:736.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:737.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692292736935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42671 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:737.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:737.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:737.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:737.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:737.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692292737402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:737.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:890.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:890.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692292890577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:890.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:891.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:891.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692292891133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:891.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:891.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:891.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:891.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:891.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692292891501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:891.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:945.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:945.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692292945584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:945.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:946.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:946.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692292946115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:946.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:946.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:946.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:946.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:946.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692292946836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:946.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:067.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:067.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692293067182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:067.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:068.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:068.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692293068157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:068.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:53:068.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:53:068.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:53:068.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:53:068.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692293068572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:53:068.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:55:030.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:030.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692295030691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:55:030.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:55:031.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:031.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692295031134 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:55:031.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:55:031.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:55:031.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:55:031.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:031.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692295031543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:55:031.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:527.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:527.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692296527674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:527.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:527.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:527.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692296527870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33717 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:527.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:528.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:528.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:528.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:528.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692296528263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:528.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:528.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:528.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692296528268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:528.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:528.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:528.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692296528356 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:528.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:528.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:528.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692296528457 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:56:528.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:529.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:529.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692296529110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:529.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:529.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:529.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:56:529.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692296529150 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:529.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:553.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:553.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692296553567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:553.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:553.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:553.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692296553825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40455 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:553.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:554.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:554.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:554.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:554.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692296554256 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:554.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:554.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:554.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692296554264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:554.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:554.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:554.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692296554337 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:554.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:554.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:554.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692296554411 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:554.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:554.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:554.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692296554425 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:56:554.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:554.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:554.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692296554890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:554.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:554.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:555.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692296554933 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:555.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:554.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:56:574.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:574.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692296574327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:574.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:574.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:574.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692296574491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:574.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:574.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:574.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:574.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:574.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692296574879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:574.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:608.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:608.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692296608071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:608.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:608.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:608.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692296608511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45257 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:608.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:608.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:608.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:608.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:608.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692296608968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:609.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:195.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:195.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692297195028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:195.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:195.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:195.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692297195390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:195.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:195.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:57:195.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:57:195.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:195.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297195801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:195.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:195.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:195.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297195807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:195.858 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:195.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:195.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297195858 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:57:195.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:57:195.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:195.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297195921 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:57:195.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:57:196.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:196.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297196454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:196.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:196.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:196.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297196523 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:57:196.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:57:196.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:57:243.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:243.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297243005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:243.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:243.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:243.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297243174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45877 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:243.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:243.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:57:243.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:57:243.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:243.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692297243547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:243.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:545.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:545.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692297545148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:545.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:545.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:545.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692297545307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:545.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:545.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:57:545.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:57:545.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:545.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692297545718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:545.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:556.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:556.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692297555995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:556.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:556.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:556.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692297556553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:556.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:57:556.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:57:556.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:57:556.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:57:556.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692297556916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:57:556.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:143.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:143.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692298143791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:143.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:144.007 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:144.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692298143950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:144.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:144.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:58:144.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:58:144.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:144.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692298144398 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:144.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:812.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692298811816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:812.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:812.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692298812093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:812.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:812.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:58:812.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:58:812.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692298812545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:812.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:812.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692298812550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:812.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:812.623 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692298812616 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:58:812.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:58:812.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.741 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692298812696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:58:812.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:58:812.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:812.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692298812711 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:58:812.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:58:813.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:58:813.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:813.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692298813258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:813.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:813.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:813.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692298813339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:58:813.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:58:858.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:858.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692298858432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:858.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:859.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:859.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692298859002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:859.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:58:859.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:58:859.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:58:859.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:58:859.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692298859421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:58:859.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:544.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:544.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692299543910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:544.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:544.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:544.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692299544637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:544.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:544.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:59:545.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:59:545.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:545.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692299545058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:545.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:627.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:627.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692299626983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:627.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:627.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:627.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692299627156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:627.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:627.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:59:627.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:59:627.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:627.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692299627687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:627.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:921.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:921.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692299921540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:921.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:921.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:921.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692299921913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:921.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:922.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:59:922.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:59:922.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:922.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692299922358 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:922.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:962.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:962.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692299962551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:962.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:963.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:963.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692299963127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43727 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:963.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:963.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:59:963.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:59:963.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:59:963.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692299963565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:59:963.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:938.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:938.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692301938552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:938.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:938.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:938.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692301938748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33469 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:938.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:939.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:01:939.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:01:939.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:939.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692301939173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:939.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:743.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:743.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303743729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:743.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:743.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:743.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303743891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:744.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:744.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:744.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:744.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:744.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303744276 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:744.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:744.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:744.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303744281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:744.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:744.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:744.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303744350 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:744.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:744.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:744.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303744393 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:03:744.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:744.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:744.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692303744859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:744.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:03:745.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:745.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:745.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692303744897 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:745.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:767.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:767.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692303767799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:767.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:768.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:768.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692303768289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:768.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:768.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:768.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:768.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:769.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303768634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:769.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:769.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:769.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303768639 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:769.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:769.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:769.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303768690 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:769.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:769.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:03:769.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:769.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303768748 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:03:769.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:769.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:769.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303769193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:769.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:769.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:769.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303769259 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:769.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:793.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:793.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692303793201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:793.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:793.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:793.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692303793752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36791 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:793.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:793.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:793.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692303793947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:793.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:794.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:794.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:794.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303794109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:794.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:794.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303794182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:794.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:794.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303794185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:794.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:794.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:794.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303794255 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:794.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:794.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:794.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692303794386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:794.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:794.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692303794603 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:794.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:794.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:794.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692303794622 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:03:794.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:795.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:795.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692303795082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:795.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:795.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:795.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692303795195 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:03:795.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:03:795.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:03:817.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:817.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692303817406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:817.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:817.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:817.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692303817552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:817.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:817.858 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:817.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:817.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:817.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692303817960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:818.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:840.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:840.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692303840333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:840.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:840.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:840.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692303840911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:840.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:841.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:03:841.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:03:841.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:03:841.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692303841813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:03:841.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:721.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:721.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692304721550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:721.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:721.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:721.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692304721741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:721.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:722.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:04:722.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:04:722.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:722.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692304722182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:722.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:799.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:799.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692304799074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:799.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:799.501 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:799.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692304799494 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40513 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:799.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:799.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:04:799.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:04:799.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:799.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692304799904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:799.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:904.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:904.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692304904001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:904.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:904.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:904.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692304904195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38279 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:904.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:904.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:04:904.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:04:904.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:04:904.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692304904610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:04:904.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:539.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:539.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692305539308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:539.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:539.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:539.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692305539475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:539.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:539.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:05:539.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:05:539.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:539.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692305539872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:539.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:539.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:539.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692305539877 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:539.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:539.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:539.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692305539942 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:05:539.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:05:540.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:540.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692305540031 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:05:540.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:05:540.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:540.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692305540050 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:05:540.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:05:540.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:540.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692305540562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:540.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:540.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:540.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692305540587 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:05:540.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:05:540.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:05:591.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:591.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692305591803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:591.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:592.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:592.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692305592247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46747 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:592.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:05:592.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:05:592.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:05:592.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:05:593.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692305592961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:05:593.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:538.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:538.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692306538109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:538.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:538.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:538.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692306538271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39823 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:538.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:538.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:538.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:538.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:538.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692306538716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:538.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:576.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:576.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692306576326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:576.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:576.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:577.007 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692306576933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43627 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:577.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:577.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:577.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:577.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:577.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692306577312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:577.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:654.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:654.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692306654045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:654.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:654.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:654.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692306654612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46475 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:654.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:654.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:654.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:655.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:655.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692306655040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:655.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:776.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:776.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692306776769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:776.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:777.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:777.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692306777376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34023 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:777.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:06:777.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:06:777.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:06:777.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:06:777.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692306777811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:06:777.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:07:662.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:662.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692307661927 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 2529 dequeues = 2529 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 2566 dequeues = 2566 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692278685104 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692306777823 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:05:07:662.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:07:662.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:662.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692307662031 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 505 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 573 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 136 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1941 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 554 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1266, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2529, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:07:662.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:07:668.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:668.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692307668254 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:05:07:668.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:05:07:668.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:668.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692307668297 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 2217751264 used = 32524 free = 0 buffers = 0 cached = 2303930040 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.05 five_min_avg = 0.17 fifteen_min_avg = 0.15 ] cpu_share = 1.51667 ] ] 2025-09-01 Mon 02:05:07:668.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:07:668.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:668.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692307668302 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.51667 used_sys_mem = 1991069780 one_min_cpuload = 0.05 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:05:07:668.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:05:07:668.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:668.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692307668327 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 761 in_bytes = 82429 out_tpkts = 819 out_bytes = 45480 exception_packets = 33 exception_packets_allowed = 22 raw_vhost_stats= [ name = vhost0 in_pkts = 761 in_bytes = 82429 out_pkts = 819 out_bytes = 45480 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 700 in_bytes = 40482 out_pkts = 763 out_bytes = 82481 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:07:668.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:07:668.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:07:668.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692307668348 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:07:668.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:08:411.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:411.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308410871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:411.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:411.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:411.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308411032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:411.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:411.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:08:411.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:08:411.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:411.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692308411496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:08:411.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:08:685.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:08:685.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:05:08:685.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:08:685.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308685312 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:08:685.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:08:685.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:08:685.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:08:685.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:08:685.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308685562 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 505 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 574 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 136 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1944 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 555 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1268, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2539, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:08:685.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:08:685.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:08:685.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:08:685.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:08:685.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Idle 2025-09-01 Mon 02:05:08:686.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:686.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308685907 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 506 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 574 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 136 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1944 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 555 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1268, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2541, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:08:686.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:08:686.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:686.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308685933 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:08:686.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:08:686.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:08:686.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:08:686.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692308686022 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 506 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 575 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 136 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1944 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 555 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1269, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2541, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:08:686.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:10:594.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692310593867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:594.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:594.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692310594033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39097 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:594.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:594.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:594.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:594.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310594588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:594.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:594.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310594593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:594.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:594.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310594643 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:594.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:594.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310594713 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:594.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:594.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:594.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310594728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:10:594.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:595.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:595.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310595285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:595.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:595.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:595.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310595324 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:595.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:595.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:10:615.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:615.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310615147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:615.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:615.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:615.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692310615669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40633 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:615.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:615.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:616.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:616.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:616.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692310616104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:616.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:616.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:616.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692310616116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:616.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:616.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:616.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310616193 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:616.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:616.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:616.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310616235 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:10:616.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:616.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:616.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692310616756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:616.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:616.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:616.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692310616789 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:616.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:616.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:10:644.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:644.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310644536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:644.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:644.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:644.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310644899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:644.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:645.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:645.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:645.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:645.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310645282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:645.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:667.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:667.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692310667043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:667.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:667.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:667.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692310667188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:667.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:667.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:667.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:667.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:667.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692310667665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:667.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:789.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:789.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692310789424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:789.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:789.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:789.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692310789917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:789.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:790.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:790.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:790.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:790.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310790345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:790.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:790.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:790.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310790350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:790.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:790.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:790.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310790397 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:790.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:790.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:790.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692310790470 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:10:790.495 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:790.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:790.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692310790937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:790.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:790.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:790.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:10:791.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692310790973 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:10:791.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:10:837.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:837.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692310837115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:837.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:837.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:837.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692310837735 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:837.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:838.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:838.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:838.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:838.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692310838361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:838.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:680.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:680.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692311680375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:680.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:680.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:680.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692311680906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:680.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:681.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:11:681.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:11:681.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:681.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692311681595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:681.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:720.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:721.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692311720813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:721.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:721.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:721.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692311721417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:721.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:721.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:11:721.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:11:721.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:721.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692311721867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:721.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:810.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:810.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692311810476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:810.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:810.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:810.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692311810875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40155 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:810.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:11:811.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:11:811.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:11:811.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:11:811.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692311811294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:11:811.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:479.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:479.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312479634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:479.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:480.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:480.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692312480669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43603 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:480.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:481.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:12:481.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:12:481.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:481.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:481.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:481.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:481.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481277 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:12:481.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:12:481.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481364 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:12:481.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:12:481.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481380 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:12:481.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:12:481.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:481.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:481.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:481.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692312481875 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:12:481.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:12:482.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:12:527.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:527.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692312527304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:527.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:527.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:527.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692312527482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:527.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:12:527.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:12:527.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:12:527.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:12:527.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692312527911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:12:527.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:553.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:553.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692313553736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:553.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:553.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:553.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692313553905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36733 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:553.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:554.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:554.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:554.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:554.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692313554332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:554.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:572.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:572.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692313572052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:572.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:572.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:572.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692313572234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:572.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:572.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:572.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:572.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:572.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692313572636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:572.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:719.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:719.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692313719455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:719.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:720.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:720.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692313720095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36831 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:720.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:720.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:720.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:720.595 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:720.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692313720585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:720.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:774.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:774.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692313773996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:774.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:774.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:774.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692313774436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38479 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:774.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:774.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:13:774.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:13:774.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:774.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692313774971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:13:774.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:058.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:45470 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:15:070.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:45476 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:15:570.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:570.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692315570816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:571.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:571.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:571.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692315571003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:571.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:15:571.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:15:571.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:15:571.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:15:571.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692315571423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:15:571.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:356.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:356.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317356709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:356.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:356.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:356.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317356865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43031 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:356.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:357.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:357.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:357.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:357.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317357287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:357.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:357.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:357.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317357297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:357.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:357.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:357.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692317357375 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:357.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:357.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:357.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692317357421 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:17:357.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:357.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:357.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692317357845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:357.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:17:357.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:357.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:357.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692317357865 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:357.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:402.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:403.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317402948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:403.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:403.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:403.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692317403424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37289 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:403.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:403.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:403.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:403.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:403.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692317403882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:403.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:406.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:406.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692317406477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:406.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:406.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:406.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692317406697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:406.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:407.024 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:407.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:407.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317407152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:407.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:407.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317407157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:407.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:407.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692317407202 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:407.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:407.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692317407285 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:407.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:407.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692317407301 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:17:407.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:407.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317407758 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:407.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:407.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:407.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:17:407.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317407796 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:407.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:454.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:454.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692317454023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:454.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:454.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:454.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692317454180 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:454.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:454.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:454.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:454.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:454.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692317454684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:454.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:483.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:483.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692317483145 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:483.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:483.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:483.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692317483510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46555 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:483.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:483.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:483.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:483.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:483.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317483947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:483.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:483.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:484.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317483952 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:484.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:484.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:484.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317484041 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:484.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:484.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:484.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317484083 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:17:484.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:484.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:484.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692317484592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:484.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:17:484.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:484.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:484.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692317484632 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:17:484.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:17:531.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:531.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692317531748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:531.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:532.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:532.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692317532298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40319 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:532.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:17:532.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:17:532.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:17:532.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:17:532.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692317532667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:17:532.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:313.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:313.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692318313001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:313.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:313.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:313.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692318313566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36327 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:313.640 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:313.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:18:313.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:18:314.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:314.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692318314032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:314.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:390.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:390.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692318390823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:390.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:391.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:391.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692318391330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:391.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:391.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:18:391.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:18:391.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:391.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692318391921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:391.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:558.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:558.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692318558801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:559.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:559.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:559.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692318559328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:559.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:18:559.640 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:18:559.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:18:559.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:18:559.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692318559737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:18:559.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:603.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:603.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692319603498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:603.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:603.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:603.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692319603675 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43173 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:603.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:604.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:19:604.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:19:604.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:604.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692319604110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:604.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:604.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:604.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692319604116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:604.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:604.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:604.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692319604165 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:19:604.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:19:604.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:604.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692319604241 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:19:604.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:19:604.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:604.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692319604370 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:19:604.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:19:605.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:605.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692319605029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:605.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:605.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:605.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692319605066 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:19:605.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:19:605.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:19:654.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:654.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692319654349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:654.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:654.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:654.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692319654822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46467 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:654.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:19:655.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:19:655.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:19:655.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:19:655.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692319655270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:19:655.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:282.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:282.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692320282182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:282.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:282.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:282.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692320282362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:282.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:282.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:282.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:282.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:282.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692320282828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:282.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:384.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:384.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692320384189 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:384.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:384.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:384.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692320384342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:384.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:384.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:384.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:384.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:385.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692320384930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:385.025 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:395.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:395.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692320395879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:395.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:396.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:396.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692320396155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:396.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:396.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:396.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:396.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:396.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692320396670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:396.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:645.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:645.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692320645441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:645.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:645.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:646.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692320645976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:646.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:646.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:646.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:646.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:646.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692320646450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:646.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:608.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:608.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692322608583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:608.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:609.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:609.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692322609415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40591 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:609.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:609.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:22:609.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:22:609.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:609.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692322609787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:609.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:201.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:201.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324201083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:201.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:201.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:201.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324201619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45899 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:201.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:201.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:201.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:202.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:202.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324202024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:202.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:202.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:202.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324202029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:202.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:202.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:202.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324202081 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:202.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:202.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:202.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324202123 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:24:202.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:202.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:202.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692324202576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:202.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:202.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:202.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692324202607 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:202.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:203.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:24:250.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:250.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692324250451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:250.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:250.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:250.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324250903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:250.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:251.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:251.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:251.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:251.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692324251305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:251.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:428.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:428.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692324428769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:428.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:429.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:429.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692324429203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:429.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:429.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:429.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:429.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:430.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324429607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:430.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:430.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:430.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324429616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:430.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:430.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:430.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324429688 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:430.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:430.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:430.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324429733 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:24:430.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:430.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:430.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324430207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:430.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:430.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:430.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324430247 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:430.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:430.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:24:479.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:479.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324479386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:479.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:479.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:480.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692324479956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:480.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:480.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:480.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:480.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:480.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692324480413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:480.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:495.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:495.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692324494981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:495.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:495.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:495.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324495432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:495.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:495.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:495.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:495.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:495.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324495845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:495.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:495.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:496.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324495849 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:496.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:496.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:496.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324495899 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:496.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:496.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:496.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324495981 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:496.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:496.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:496.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692324495996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:24:496.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:496.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:496.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692324496471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:496.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:496.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:496.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692324496516 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:24:496.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:24:496.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:24:544.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:544.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692324544830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:544.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:545.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:545.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692324545367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45501 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:545.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:545.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:24:545.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:24:546.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:546.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692324546025 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:24:546.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:268.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:268.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692325268424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:268.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:268.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:268.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692325268600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:268.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:268.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:25:268.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:25:269.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:269.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692325269062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:269.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:398.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:398.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692325398581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:398.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:398.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:398.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692325398821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33659 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:398.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:399.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:25:399.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:25:399.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:399.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692325399288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:399.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:527.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:527.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692325527151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:527.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:527.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:527.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692325527699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:527.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:528.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:25:528.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:25:528.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:528.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692325528136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:528.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:282.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692326281986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:282.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:282.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692326282181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:282.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:282.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:26:282.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:26:282.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692326282563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:282.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:282.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692326282567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:282.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:282.640 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692326282635 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:26:282.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:26:282.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692326282732 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:26:282.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:26:282.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:282.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692326282751 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:26:282.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:26:283.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:283.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692326283339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:283.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:283.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:283.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692326283387 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:26:283.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:26:283.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:26:329.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:329.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692326329730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:329.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:330.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:330.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692326330156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:330.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:26:330.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:26:330.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:26:330.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:330.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692326330597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:26:330.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:087.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:087.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692327087199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:087.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:087.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:087.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692327087765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35025 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:087.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:088.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:088.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:088.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:088.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692327088199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:088.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:248.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:248.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692327248744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:248.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:249.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:249.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692327249449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:249.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:249.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:249.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:249.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:249.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692327249873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:249.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:253.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:253.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692327253478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:253.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:253.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:254.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692327253985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32773 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:254.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:254.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:254.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:254.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:254.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692327254393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:254.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:508.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:508.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692327508263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:508.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:508.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:508.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692327508755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34205 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:508.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:509.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:509.097 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:509.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:509.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692327509122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:509.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:050.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:29:050.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692329050024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:29:050.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:051.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:29:051.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692329051032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:29:051.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:051.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:29:051.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:29:051.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:29:051.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692329051420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:29:051.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:688.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:688.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692330688325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:688.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:688.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:688.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692330688893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:688.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:689.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:689.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:689.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:689.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692330689603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:689.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:689.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:689.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692330689608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:689.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:689.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:689.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692330689652 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:689.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:689.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:689.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692330689697 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:30:689.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:690.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:690.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692330690134 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:690.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:690.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:690.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692330690178 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:690.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:690.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:30:735.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:735.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692330735547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:735.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:735.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:735.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692330735945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43377 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:735.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:736.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:736.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:736.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:736.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692330736378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:736.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:962.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:962.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692330962548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:962.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:964.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:964.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692330964073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33775 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:964.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:965.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:965.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:965.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:965.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692330965507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:965.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:965.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:965.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692330965512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:965.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:965.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:965.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692330965677 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:965.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:965.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:965.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692330965732 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:30:965.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:30:966.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:966.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:30:966.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692330966249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:966.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:966.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:966.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692330966274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:30:966.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:014.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:014.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692331014477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:014.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:015.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:015.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692331015048 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:015.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:015.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:015.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:015.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:015.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692331015473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:015.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:145.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:145.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692331145284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:145.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:145.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:145.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692331145796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:145.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:146.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:146.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:146.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:146.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692331146276 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:146.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:146.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:146.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692331146282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:146.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:146.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:146.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692331146330 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:31:146.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:146.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:146.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692331146404 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:31:146.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:146.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:146.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692331146419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:31:146.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:147.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:147.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692331146995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:147.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:147.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:147.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692331147043 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:31:147.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:31:147.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:31:197.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:197.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692331197297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:197.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:197.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:197.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692331197895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42495 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:197.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:198.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:198.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:198.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:198.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692331198230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:198.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:807.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:807.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692331807508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:807.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:808.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:808.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692331808001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40043 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:808.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:808.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:31:808.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:31:808.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:31:808.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692331808401 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:31:808.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:041.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:041.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692332041623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:041.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:042.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:042.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692332042198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:042.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:042.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:32:042.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:32:042.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:042.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692332042628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:042.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:206.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:206.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692332206366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:206.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:207.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:207.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692332206994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:207.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:32:207.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:32:207.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:32:207.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:32:207.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692332207388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:32:207.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:047.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:047.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692333047554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:047.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:048.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:048.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692333048482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:048.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:048.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:33:049.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:33:049.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:049.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:049.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:049.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:049.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:049.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:049.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:049.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049290 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:33:050.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:33:050.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:050.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049396 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:33:050.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:33:050.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:050.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049414 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:33:050.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:33:050.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:050.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:050.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:050.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:050.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333049935 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:33:050.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:33:050.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:33:095.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:095.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692333095088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:095.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:095.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:095.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692333095836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:095.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:096.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:33:096.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:33:096.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:096.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692333096262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:096.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:770.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:770.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692333770523 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:770.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:771.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:771.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692333771413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:771.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:771.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:33:771.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:33:771.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:771.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692333771764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:771.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:996.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:996.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692333996396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:996.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:996.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:997.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692333996954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:997.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:33:997.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:33:997.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:33:997.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:33:997.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692333997479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:33:997.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:184.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:184.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692334184759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:184.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:185.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:185.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692334185205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41339 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:185.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:185.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:185.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:185.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:185.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692334185650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:185.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:295.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:295.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692334295533 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:295.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:295.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:295.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692334295902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:295.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:34:296.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:34:296.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:34:296.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:34:296.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692334296334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:34:296.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:995.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:995.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692335995615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:995.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:996.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:996.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692335996155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:996.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:35:996.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:35:996.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:35:996.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:35:996.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692335996710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:35:996.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:662.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:662.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692337662165 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 2824 dequeues = 2824 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 2864 dequeues = 2864 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692308685977 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692335996715 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:05:37:662.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:37:662.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:662.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692337662279 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 563 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 641 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2168 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 619 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1415, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2824, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:37:662.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:37:668.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:668.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337668476 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:05:37:668.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:05:37:668.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:668.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337668532 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 858 in_bytes = 99259 out_tpkts = 919 out_bytes = 51234 exception_packets = 35 exception_packets_allowed = 24 raw_vhost_stats= [ name = vhost0 in_pkts = 858 in_bytes = 99259 out_pkts = 919 out_bytes = 51234 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] total_in_bandwidth_utilization = 6.67326e-15 total_out_bandwidth_utilization = 1.65666e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1231, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3056, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 785 in_bytes = 45606 out_pkts = 860 out_bytes = 99311 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:37:668.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:37:668.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:668.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337668589 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:37:668.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:37:785.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:785.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337785783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:785.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:785.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:785.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337785923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:785.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:786.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:786.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:786.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:786.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337786331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:787.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:787.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:787.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337786335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:787.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:787.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:787.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337786390 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:787.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:787.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:787.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337786436 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:37:787.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:787.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:787.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337786914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:787.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:787.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:787.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692337786953 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:787.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:786.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:37:838.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:838.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692337838142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:838.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:838.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:838.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692337838308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41239 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:838.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:838.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:838.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:838.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:838.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337838752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:838.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:984.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:984.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692337983892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:984.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:984.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:984.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692337984393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:984.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:984.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:37:984.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:37:984.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:985.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337984827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:985.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:985.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:985.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337984838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:985.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:985.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:985.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337984899 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:985.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:985.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:985.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337984944 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:37:985.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:985.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:985.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337985394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:37:985.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:37:985.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:37:985.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692337985481 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:37:985.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:37:985.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:38:036.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:036.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692338036648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:036.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:037.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:037.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692338037086 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:037.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:037.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:037.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:037.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:037.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692338037722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:037.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:400.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:400.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692338400462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:400.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:401.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:401.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692338401115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:401.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:401.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:401.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:401.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:401.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692338401507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:401.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:401.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:401.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692338401515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:401.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:401.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:401.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692338401570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:401.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:401.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:401.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692338401822 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:401.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:401.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:401.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692338401838 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:38:401.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:402.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:402.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692338402341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:402.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:402.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:402.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692338402382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:402.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:402.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:38:450.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:450.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692338450584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:450.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:450.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:450.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692338450738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:450.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:451.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:451.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:451.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:451.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692338451168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:451.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:686.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:38:686.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:05:38:686.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:38:686.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692338686321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:686.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:686.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:38:687.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:38:687.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:38:687.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692338686863 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 573 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 155 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 625 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1430, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2863, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:38:687.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:38:687.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:38:687.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:38:687.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:687.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Idle 2025-09-01 Mon 02:05:38:687.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:687.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692338687270 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 574 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 155 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 625 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1430, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2865, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:38:687.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:38:687.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:687.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692338687295 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:38:687.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:38:687.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:38:687.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:687.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692338687386 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 574 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 155 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 625 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1431, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2865, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:38:687.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:38:747.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:747.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692338747891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:747.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:748.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:748.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692338748447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:748.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:748.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:748.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:748.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:748.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692338748844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:748.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:946.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:946.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692338946896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:946.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:947.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:947.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692338947289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43125 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:947.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:947.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:947.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:947.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:947.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692338947926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:947.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:39:522.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:39:522.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692339522318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:39:522.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:39:522.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:39:522.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692339522707 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38567 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:39:522.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:39:523.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:39:523.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:39:523.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:39:523.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692339523297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:39:523.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:172.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:172.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692340172847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:173.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:173.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:173.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692340173640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:173.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:173.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:40:173.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:40:174.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:174.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:174.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:174.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:174.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174151 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:40:174.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:174.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174253 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:40:174.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:174.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174273 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:40:174.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:174.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:174.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:174.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:174.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340174797 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:40:174.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:40:174.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:40:224.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:224.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692340223932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:224.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:224.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:224.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692340224813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:224.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:225.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:40:225.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:40:225.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:225.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692340225192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:225.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:837.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:837.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692340836983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:837.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:838.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:838.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692340838545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:838.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:40:838.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:40:838.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:40:839.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:40:839.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692340839018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:40:839.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:018.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:018.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692341018055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:018.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:018.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:019.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692341018958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44915 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:019.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:019.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:019.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:019.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:019.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692341019283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:019.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:233.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:233.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692341233399 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:233.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:235.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:235.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692341234997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45281 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:235.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:235.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:235.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:235.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:235.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692341235335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:235.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:611.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:611.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692341611457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:611.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:612.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:612.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692341612077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:612.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:612.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:41:612.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:41:612.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:41:612.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692341612463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:41:612.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:43:107.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:107.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692343107497 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:43:107.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:43:108.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:108.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692343108147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:43:108.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:43:108.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:43:108.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:43:108.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:108.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692343108615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:43:108.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:511.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:511.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692344511167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:511.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:511.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:511.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344511634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37865 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:511.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:511.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:511.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:512.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:512.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692344512036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:512.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:512.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:512.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692344512042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:512.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:512.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:512.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692344512112 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:512.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:512.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:512.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692344512154 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:44:512.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:512.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:512.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692344512654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:512.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:512.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:512.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692344512688 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:512.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:512.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:44:566.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:566.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344566065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:566.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:566.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:566.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692344566600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:566.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:566.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:566.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:566.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:567.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692344566964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:567.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:799.507 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:799.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344799404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:799.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:799.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:799.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692344799880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:799.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:800.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:800.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:800.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:800.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344800226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:800.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:800.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:800.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344800232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:800.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:800.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:800.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344800313 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:800.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:800.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:800.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344800359 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:44:800.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:800.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:800.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692344800827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:800.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:800.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:800.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692344800852 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:800.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:800.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:44:848.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:848.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692344847991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:848.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:848.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:848.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692344848404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45079 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:848.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:44:848.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:44:848.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:848.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:848.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692344848808 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:44:848.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:530.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:530.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345530114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:530.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:530.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:530.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345530635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41907 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:530.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:530.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:530.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:531.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:531.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692345531002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:531.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:824.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:824.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345824608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:824.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:825.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:825.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692345825056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:825.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:825.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:825.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:825.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:825.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345825646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:825.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:825.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:825.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345825651 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:825.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:825.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:825.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345825697 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:45:825.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:825.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:825.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345825771 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:45:825.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:825.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:825.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692345825790 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:45:825.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:826.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:826.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345826277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:826.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:826.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:826.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345826319 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:45:826.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:45:826.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:45:838.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:838.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345838878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:838.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:839.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:839.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692345839178 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36799 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:839.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:839.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:839.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:839.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:839.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345839533 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:839.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:871.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:871.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345871578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:871.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:872.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:872.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692345872096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:872.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:45:872.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:45:872.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:45:872.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:45:872.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692345872536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:45:872.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:46:889.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:46:889.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692346889660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:46:889.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:46:890.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:46:890.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692346890161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:46:890.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:46:890.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:46:890.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:46:890.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:46:890.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692346890660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:46:890.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:176.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:176.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692347176771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:176.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:177.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:177.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692347177380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43281 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:177.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:177.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:177.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:177.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:177.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692347177742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:177.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:177.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:177.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692347177750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:177.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:177.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:177.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692347177802 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:47:178.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:178.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:178.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692347178071 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:47:178.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:178.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:178.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692347178087 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:47:178.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:178.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:178.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692347178635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:178.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:178.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:178.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692347178664 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:47:178.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:47:178.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:47:225.858 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:225.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692347225816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:225.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:226.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:226.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692347226393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:226.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:226.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:226.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:226.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:226.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692347226827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:226.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:439.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:439.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692347439123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:439.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:439.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:439.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692347439700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:439.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:440.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:440.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:440.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:440.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692347440286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:440.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:819.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:819.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692347819647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:819.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:820.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:820.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692347820170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:820.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:47:820.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:47:820.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:47:820.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:47:820.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692347820629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:47:820.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:190.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:190.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692348189949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:191.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:191.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:191.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:191.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692348190684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:191.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:191.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:191.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:191.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692348191839 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:191.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:816.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:817.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692348816793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:817.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:817.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:817.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692348817452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:817.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:817.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:817.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:817.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:817.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692348817865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:817.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:50:190.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:50:190.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692350189894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:50:190.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:50:190.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:50:190.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692350190054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:50:190.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:50:190.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:50:190.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:50:190.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:50:190.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692350190462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:50:190.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:220.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:220.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692351220419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:220.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:221.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:221.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692351221001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42985 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:221.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:221.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:221.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:221.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:221.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692351221481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:221.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:221.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:221.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692351221485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:221.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:221.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:221.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692351221534 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:221.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:221.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:221.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692351221576 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:51:221.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:222.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:222.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692351222056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:222.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:222.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:222.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692351222081 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:222.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:222.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:51:271.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:271.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692351271363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:271.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:271.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:271.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692351271757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36401 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:271.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:272.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:272.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:272.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:272.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692351272334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:272.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:636.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692351636693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:637.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:637.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692351636863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:637.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:637.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:637.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:637.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692351637377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:637.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:637.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692351637384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:637.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:637.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692351637435 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:637.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:637.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692351637477 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:51:637.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:637.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:637.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692351637908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:637.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:638.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:638.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692351637934 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:51:638.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:51:638.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:51:686.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:686.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692351686295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:686.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:686.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:686.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692351686899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:686.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:51:687.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:51:687.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:51:687.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:51:687.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692351687236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:51:687.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:253.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:253.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352253482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:253.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:254.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:254.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692352254057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:254.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:254.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:254.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:254.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:254.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352254585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:254.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:742.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:742.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692352741977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:742.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:743.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:743.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692352743016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35827 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:743.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:743.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:743.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:744.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352743362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:744.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:744.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352743367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:744.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:744.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352743447 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:52:744.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:744.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352743533 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:52:744.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:744.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352743548 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:52:744.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:744.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352744038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:744.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:744.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:744.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352744133 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:52:744.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:52:744.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:52:749.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:749.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692352749340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:749.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:749.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:749.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692352749734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:749.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:750.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:750.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:750.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:750.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692352750313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:750.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:790.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:790.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692352790293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:790.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:790.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:790.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692352790820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:790.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:52:791.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:52:791.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:52:791.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:52:791.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692352791273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:52:791.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:772.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:772.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692353772434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:772.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:772.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:772.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692353772840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:772.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:773.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:53:773.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:53:773.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:773.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692353773233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:773.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:898.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:898.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692353898544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:898.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:898.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:898.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692353898729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:898.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:899.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:53:899.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:53:899.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:899.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692353899159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:899.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:899.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:899.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692353899181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:899.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:899.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:899.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692353899238 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:53:899.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:53:899.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:899.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692353899482 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:53:899.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:53:899.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:899.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692353899498 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:53:899.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:53:899.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:899.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692353899971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:899.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:900.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:900.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:53:900.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692353899996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:53:900.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:53:947.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:947.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692353947121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:947.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:947.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:947.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692353947302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:947.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:947.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:53:947.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:53:947.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:947.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692353947708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:947.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:198.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:198.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692354198708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:198.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:198.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:198.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692354198876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:198.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:199.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:199.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:199.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:199.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692354199295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:199.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:784.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:784.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692354784459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:784.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:785.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:785.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692354785185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43631 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:785.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:785.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:785.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:785.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:785.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692354785633 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:785.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:964.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:965.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692354964849 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:965.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:965.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:965.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692354965434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:965.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:965.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:965.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:965.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:965.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692354965924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:965.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:573.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:573.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692355573372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:573.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:573.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:573.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692355573784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:573.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:574.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:55:574.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:55:574.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:574.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692355574244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:55:574.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:000.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:000.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692357000050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:000.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:000.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:000.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692357000206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:000.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:000.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:57:000.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:57:000.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:000.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692357000598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:000.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:303.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:303.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692358303368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:303.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:303.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:303.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358303807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46055 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:303.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:304.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:58:304.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:58:304.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:304.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358304351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:304.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:304.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:304.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358304355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:304.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:304.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:304.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358304416 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:304.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:304.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:304.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358304457 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:58:304.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:304.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:304.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358304932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:304.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:305.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:305.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358304984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:305.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:305.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:58:351.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:351.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358351313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:351.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:351.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:351.858 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358351834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:351.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:352.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:58:352.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:58:352.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:352.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358352315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:352.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:961.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:961.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358961766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:963.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:962.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:58:963.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:963.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692358962412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32877 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:963.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:963.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:58:963.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:963.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692358963453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:963.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:963.501 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:963.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692358963459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:963.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:963.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:963.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692358963510 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:963.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:963.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:963.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692358963556 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:58:963.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:964.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:964.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358964017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:58:964.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:58:964.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:58:964.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692358964048 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:58:964.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:58:964.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:59:014.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:014.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692359014410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:014.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:014.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:014.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692359014947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36151 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:014.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:015.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:015.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:015.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:015.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692359015801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:015.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:333.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:333.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692359333448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:333.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:334.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:334.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692359334333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:334.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:334.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:334.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:334.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:334.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692359334706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:334.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:390.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:390.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359390364 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:390.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:391.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:391.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692359391237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:391.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:391.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:391.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:391.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:391.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359391575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:391.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:391.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:391.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359391579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:391.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:392.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:392.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359391707 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:59:392.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:59:392.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:392.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359391794 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:59:392.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:59:392.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:392.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359391808 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:05:59:392.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:59:392.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:392.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359392290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:392.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:392.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:392.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692359392316 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:59:392.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:59:392.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:05:59:443.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:443.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692359443641 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:443.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:443.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:443.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692359443896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33941 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:443.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:444.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:444.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:444.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:444.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692359444330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:444.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:925.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:925.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692359924939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:925.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:925.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:925.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692359925400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37969 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:925.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:925.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:925.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:925.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:925.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692359925834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:925.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:00:434.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:00:434.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692360434435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:00:434.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:00:435.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:00:435.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692360435024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:00:435.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:00:435.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:00:435.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:00:435.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:00:435.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692360435518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:00:435.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:176.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:176.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692361176698 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:176.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:177.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:177.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692361177151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34139 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:177.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:177.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:177.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:177.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:177.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692361177610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:177.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:177.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:177.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692361177615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:177.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:177.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:177.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692361177662 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:01:177.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:177.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:177.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692361177736 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:01:177.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:177.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:177.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692361177751 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:01:177.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:178.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:178.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692361178297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:178.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:178.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:178.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692361178341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:01:178.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:01:178.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:01:231.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:232.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692361231838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:232.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:232.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:232.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692361232331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:232.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:232.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:232.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:232.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:232.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692361232798 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:232.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:440.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:441.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692361440847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:441.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:441.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:441.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692361441444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42275 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:441.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:441.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:441.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:442.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:442.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692361441999 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:442.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:726.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:726.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692361725973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:726.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:726.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:726.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692361726596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41085 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:726.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:01:726.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:01:727.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:01:727.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:01:727.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692361727210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:01:727.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:160.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:160.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692362160037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:160.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:160.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:160.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692362160224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:160.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:160.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:02:160.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:160.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:160.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692362160640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:160.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:289.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:289.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692362289686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:289.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:290.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:290.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692362290131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:290.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:02:290.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:02:290.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:02:290.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:02:290.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692362290566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:02:290.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:03:978.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:03:978.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692363978769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:03:978.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:03:979.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:03:979.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692363979350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:03:979.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:03:979.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:03:979.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:03:979.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:03:979.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692363979669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:03:979.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:222.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:222.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365222148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:222.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:223.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:223.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692365223039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38609 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:223.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:223.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:223.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:223.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:223.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365223686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:223.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:223.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:223.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365223690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:223.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:224.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:224.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365223821 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:224.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:224.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:224.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365223861 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:05:224.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:224.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:224.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365224340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:224.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:224.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:224.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365224365 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:224.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:224.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:05:270.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:270.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365270686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:270.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:271.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:271.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365271117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:271.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:271.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:271.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:271.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:271.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692365271743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:271.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:543.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:543.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365543352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:543.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:543.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:543.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692365543885 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39901 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:543.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:544.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:544.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:544.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:544.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365544462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:544.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:544.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:544.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365544467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:544.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:544.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:544.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365544550 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:544.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:544.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:544.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365544611 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:05:544.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:545.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:545.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365545065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:545.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:545.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:545.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365545101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:05:545.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:05:545.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:05:590.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:590.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365590507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:590.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:590.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:590.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692365590912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:590.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:591.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:05:591.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:05:591.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:05:591.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692365591352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:05:591.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:070.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:070.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692366070700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:070.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:071.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:071.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692366071216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:071.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:071.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:071.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:071.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:071.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692366071706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:071.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:071.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:071.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692366071711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:071.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:071.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:071.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692366071765 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:06:071.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:071.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:071.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692366071859 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:06:071.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:071.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:071.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692366071875 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:06:071.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:072.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:072.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692366072352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:072.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:072.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:072.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692366072388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:06:072.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:06:072.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:06:122.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:122.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692366122588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:122.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:122.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:122.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692366122764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44205 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:122.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:123.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:123.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:123.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:123.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692366123161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:123.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:226.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:226.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692366225950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:226.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:226.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:226.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692366226163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:226.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:226.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:226.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:226.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:226.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692366226569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:226.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:509.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:509.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692366509475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:509.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:509.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:510.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692366509968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42071 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:510.022 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:06:510.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:06:510.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:06:510.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:06:510.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692366510369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:06:510.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:032.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:032.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692367032282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:07:032.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:032.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:032.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692367032845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43837 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:07:032.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:033.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:07:033.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:07:033.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:033.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692367033224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:07:033.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:07:662.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:662.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692367662383 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 3157 dequeues = 3157 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 3200 dequeues = 3200 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692338687352 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692367033233 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:06:07:662.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:07:662.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:662.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692367662502 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 631 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 715 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 171 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2425 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 692 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1581, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3157, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:07:662.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:07:668.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:668.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692367668657 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:06:07:668.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:06:07:668.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:668.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692367668693 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 2213552864 used = 32524 free = 0 buffers = 0 cached = 2303930040 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.015 five_min_avg = 0.135 fifteen_min_avg = 0.14 ] cpu_share = 1.53308 ] ] 2025-09-01 Mon 02:06:07:668.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:07:668.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:668.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692367668699 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.53308 used_sys_mem = 1991069780 one_min_cpuload = 0.015 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:06:07:668.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:06:07:668.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:668.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692367668721 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 950 in_bytes = 105459 out_tpkts = 1012 out_bytes = 56044 exception_packets = 39 exception_packets_dropped = 12 exception_packets_allowed = 27 raw_vhost_stats= [ name = vhost0 in_pkts = 950 in_bytes = 105459 out_pkts = 1012 out_bytes = 56044 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 865 in_bytes = 49870 out_pkts = 952 out_bytes = 105511 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:07:668.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:07:668.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:07:669.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692367668980 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:07:669.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:08:062.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:062.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692368062687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:062.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:063.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:063.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692368063556 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:063.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:063.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:063.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:063.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:063.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692368063925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:063.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:227.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:227.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692368227821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:228.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:228.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:228.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692368228702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:228.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:228.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:229.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:229.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692368229109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:229.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:229.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692368229119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:229.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:229.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692368229196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:229.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:229.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692368229321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:229.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:229.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692368229343 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:08:229.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:229.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692368229806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:229.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:229.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:229.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692368229828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:229.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:229.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:08:283.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:283.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692368282998 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:283.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:283.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:283.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692368283686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43741 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:283.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:284.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:284.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:284.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:284.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692368284187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:284.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:562.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:562.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692368562478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:562.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:563.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:563.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692368562996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:563.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:563.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:563.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:563.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:563.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692368563525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:563.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:687.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:08:688.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:06:08:688.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:08:688.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692368687532 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:688.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:688.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:08:688.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:08:688.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:08:688.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692368688112 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 635 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 719 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 172 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2439 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 696 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1590, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3182, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:08:688.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:08:688.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:08:688.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:08:688.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:688.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Idle 2025-09-01 Mon 02:06:08:688.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:688.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692368688445 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 636 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 719 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 172 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2439 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 696 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1590, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3184, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:08:688.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:08:688.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:688.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692368688469 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:08:688.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:08:688.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:08:688.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:688.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692368688566 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 636 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 720 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 172 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2439 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 696 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1591, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3184, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:08:688.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:08:869.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:869.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692368869370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:869.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:869.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:869.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692368869802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36705 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:869.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:870.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:08:870.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:08:870.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:08:870.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692368870513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:08:870.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:229.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:229.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692369229348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:229.640 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:230.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:230.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692369229965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40013 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:230.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:09:230.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:09:230.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:09:230.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:09:230.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692369230510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:09:230.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:030.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:030.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692371030620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:030.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:031.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:031.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692371031195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32927 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:031.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:031.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:11:031.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:11:031.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:031.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692371031700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:031.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:844.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:844.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692371844063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:844.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:844.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:844.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692371844598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37203 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:844.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:844.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:11:845.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:11:845.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:845.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692371845224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:845.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:845.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:845.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692371845228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:845.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:845.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:845.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692371845274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:11:845.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:11:845.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:845.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692371845315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:11:845.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:11:845.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:845.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692371845880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:845.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:845.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:845.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692371845934 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:11:845.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:11:846.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:11:896.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:896.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692371896212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:896.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:896.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:896.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692371896794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:896.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:897.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:11:897.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:11:897.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:897.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692371897245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:897.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:703.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:703.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692372703659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:703.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:704.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:704.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692372704645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:704.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:704.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:705.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:705.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:705.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692372705192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:705.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:705.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:705.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692372705198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:705.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:705.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:705.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692372705249 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:705.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:705.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:705.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692372705292 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:12:705.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:705.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:705.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692372705789 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:705.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:705.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:705.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692372705836 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:705.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:706.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:12:722.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:722.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692372722651 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:722.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:722.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:722.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692372722810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34361 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:722.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:723.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:723.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:723.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692372723183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:723.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:723.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692372723188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:723.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:723.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692372723255 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:723.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:723.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692372723339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:723.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:723.389 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692372723356 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:12:723.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:723.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692372723847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:723.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:723.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:723.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692372723880 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:12:723.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:12:723.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:12:755.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:755.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692372755235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:755.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:755.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:755.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692372755751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42945 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:755.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:756.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:756.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:756.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:756.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692372756183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:756.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:771.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:771.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692372771016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:771.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:771.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:771.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692372771543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:771.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:771.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:771.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:771.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:771.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692372771918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:771.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:842.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:842.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692372842388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:842.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:842.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:842.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692372842537 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35483 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:842.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:12:842.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:12:842.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:12:842.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:12:842.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692372842899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:12:842.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:690.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:690.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692373690048 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:690.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:690.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:690.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692373690602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:690.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:690.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:13:690.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:13:691.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:691.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692373690994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:691.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:791.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:791.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692373791305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:791.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:791.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:791.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692373791848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:791.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:792.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:13:792.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:13:792.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:792.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692373792363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:13:792.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:14:661.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:661.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692374660971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:14:661.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:14:661.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:662.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692374661972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44113 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:14:662.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:14:662.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:14:662.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:14:662.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:662.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692374662335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:14:662.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:316.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375315876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:316.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:316.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375316043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:316.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:316.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:316.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:316.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375316461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:316.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:316.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375316466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:316.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:316.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375316552 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:15:316.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:316.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375316626 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:15:316.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:316.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:316.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375316641 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:15:316.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:317.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:317.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692375317181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:317.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:15:317.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:317.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:317.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692375317223 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:15:317.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:15:366.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:366.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692375366357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:366.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:366.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:366.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692375366539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34871 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:366.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:366.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:366.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:366.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:366.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692375366904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:366.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:797.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:797.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692375797184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:797.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:797.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:797.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692375797340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:797.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:797.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:797.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:797.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:797.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692375797708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:797.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:883.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:54862 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:15:895.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:54866 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:15:898.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:898.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692375898489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:898.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:899.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:899.026 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692375898990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:899.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:899.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:899.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:899.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:899.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692375899288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:899.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:276.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:276.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692376276088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:276.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:276.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:276.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692376276754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:276.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:16:277.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:16:277.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:16:277.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:16:277.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692376277152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:16:277.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:149.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:149.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692378149329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:149.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:149.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:149.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692378149490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:149.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:149.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:18:149.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:18:149.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:149.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692378149921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:149.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:766.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:766.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692378766449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:766.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:766.623 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:766.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692378766614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:766.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:766.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:18:767.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:18:767.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:767.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692378767074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:767.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:767.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:767.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692378767093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:767.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:767.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:767.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692378767157 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:18:767.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:18:767.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:767.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692378767203 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:18:767.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:18:767.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:767.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692378767692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:767.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:767.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:767.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692378767722 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:18:767.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:18:767.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:18:819.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:819.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692378819100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:819.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:819.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:819.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692378819610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34573 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:819.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:18:819.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:18:819.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:18:820.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:18:820.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692378820057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:18:820.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:837.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:837.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692379837198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:837.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:837.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:837.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692379837384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41377 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:837.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:837.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:19:837.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:19:837.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:837.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692379837896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:837.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:973.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:973.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692379973860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:973.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:974.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:974.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692379974101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:974.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:974.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:19:974.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:19:974.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:974.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692379974470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:974.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:974.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:974.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692379974478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:974.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:974.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:974.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692379974545 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:19:974.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:974.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:974.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692379974630 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:19:974.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:974.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:974.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692379974646 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:19:974.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:975.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:975.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692379975116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:19:975.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:19:975.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:19:975.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692379975151 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:19:975.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:19:975.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:20:026.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:026.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692380026297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:026.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:026.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:026.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692380026458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:026.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:026.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:20:026.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:20:026.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:026.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692380026927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:026.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:039.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:039.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692380039409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:039.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:039.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:040.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692380039967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:040.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:040.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:20:040.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:20:040.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:040.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692380040373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:040.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:040.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:040.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692380040381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:040.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:040.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:040.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692380040463 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:20:040.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:20:040.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:040.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692380040528 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:20:040.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:20:041.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:041.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692380041174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:041.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:041.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:041.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692380041210 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:20:041.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:20:041.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:20:088.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:088.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692380088513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:088.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:088.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:088.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692380088671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45507 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:088.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:20:088.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:20:088.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:20:089.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:20:089.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692380089062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:20:089.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:071.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:071.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692381071044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:071.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:071.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:071.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692381071257 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43099 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:071.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:071.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:071.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:071.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:071.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692381071697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:071.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:097.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:097.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692381097241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:097.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:097.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:097.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381097512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:097.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:097.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:097.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:097.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:097.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692381097910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:097.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:894.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692381894094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:894.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:894.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692381894258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:894.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:894.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:894.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:894.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381894728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:894.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:894.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381894733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:894.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:894.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381894800 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:21:894.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:21:894.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381894883 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:21:894.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:21:894.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:894.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381894922 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:21:894.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:21:895.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:895.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692381895388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:895.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:895.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:21:895.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:895.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692381895430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:21:895.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:21:941.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:941.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692381941574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:941.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:941.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:941.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692381941761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45537 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:941.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:942.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:942.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:942.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:942.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692381942144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:942.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:980.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:980.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692381980213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:980.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:980.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:980.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692381980820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37039 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:980.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:981.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:981.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:981.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:981.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692381981201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:981.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:997.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:998.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692382997796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:998.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:998.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:998.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692382998760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42293 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:998.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:22:999.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:22:999.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:22:999.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:22:999.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692382999286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:22:999.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:013.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:013.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692383013231 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:013.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:014.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:014.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692383014254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43427 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:014.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:014.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:014.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:014.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:014.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692383014802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:014.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:078.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:078.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692383078049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:078.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:078.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:078.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692383078668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:078.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:078.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:079.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:079.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:079.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692383079175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:079.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:031.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:031.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692385030997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:031.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:031.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:031.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692385031166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:031.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:031.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:031.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:031.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:031.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692385031527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:031.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:833.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:833.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692385833370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:833.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:833.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:833.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692385833527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39833 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:833.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:833.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:833.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:833.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:834.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385833969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:834.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:834.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:834.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385833972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:834.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:834.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:834.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385834037 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:834.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:834.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:834.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385834080 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:25:834.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:834.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:834.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385834642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:834.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:834.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:834.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385834693 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:834.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:834.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:25:884.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:884.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385884820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:884.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:885.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:885.027 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692385884998 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:885.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:885.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:885.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:885.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:885.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692385885458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:885.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:920.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:920.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692386920598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:920.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:921.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:921.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692386921116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44575 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:921.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:26:921.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:26:921.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:26:921.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:921.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692386921536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:26:921.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:031.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:031.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692387031422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:031.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:031.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:032.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692387031959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42375 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:032.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:032.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:27:032.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:27:032.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:032.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692387032407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:032.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:032.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:032.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692387032431 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:032.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:032.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:032.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692387032483 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:032.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:032.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:032.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692387032715 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:032.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:032.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:032.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692387032733 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:27:032.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:033.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:033.276 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692387033255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:033.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:033.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:033.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692387033281 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:033.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:033.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:27:082.495 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:082.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692387082464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:082.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:082.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:082.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692387082637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40529 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:082.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:082.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:27:082.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:27:083.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:083.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692387082996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:083.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:327.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:327.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692387327324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:327.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:327.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:328.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692387327928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:328.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:328.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:27:328.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:27:328.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:328.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692387328329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:328.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:328.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:328.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692387328335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:328.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:328.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:328.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692387328415 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:328.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:328.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:328.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692387328463 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:27:328.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:329.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:329.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692387328969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:329.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:329.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:329.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692387329033 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:27:329.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:27:329.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:27:382.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:382.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692387382100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:382.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:382.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:382.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692387382476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:382.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:27:382.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:27:382.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:27:382.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:27:382.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692387382951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:27:382.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:028.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:028.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692388028175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:028.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:028.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:028.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692388028329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35925 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:028.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:028.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:28:028.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:28:028.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:028.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692388028865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:028.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:409.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:409.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692388409115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:409.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:409.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:409.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692388409495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:409.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:409.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:28:409.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:28:410.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:410.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692388410032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:410.097 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:793.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:793.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692388793687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:793.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:793.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:793.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692388793860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:793.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:794.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:28:794.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:28:794.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:794.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692388794270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:794.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:027.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:027.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692389027714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:027.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:027.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:027.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692389027869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41395 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:027.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:028.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:29:028.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:29:028.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:028.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389028282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:028.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:028.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:028.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389028287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:028.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:028.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:028.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389028351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:29:028.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:028.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:028.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389028430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:29:028.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:028.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:028.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389028445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:29:028.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:029.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:029.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:29:029.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389029183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:029.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:029.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:029.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389029223 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:29:029.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:29:080.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:080.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692389080357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:080.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:080.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:080.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692389080603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37621 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:080.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:080.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:29:081.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:29:081.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:081.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692389081117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:081.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:955.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:955.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692389954977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:955.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:955.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:955.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692389955717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:955.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:956.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:29:956.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:29:956.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:29:956.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692389956152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:29:956.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:062.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:062.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692390062282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:062.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:062.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:062.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692390062863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38467 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:062.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:063.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:30:063.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:30:063.507 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:063.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692390063491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:063.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:516.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:516.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692390516167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:516.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:517.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:517.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692390517075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33639 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:517.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:30:517.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:30:517.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:30:517.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:30:517.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692390517484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:30:517.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:169.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:169.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392169676 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:169.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:169.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:169.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392169834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:169.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:170.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:32:170.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:32:170.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:170.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392170339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:170.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:754.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:754.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692392754394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:754.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:755.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:755.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692392755363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:755.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:755.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:32:755.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:32:755.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:755.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392755705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:755.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:755.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:755.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392755709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:755.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:756.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:756.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392755832 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:32:756.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:32:756.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:756.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392755872 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:32:756.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:32:756.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:756.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392756324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:756.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:756.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:756.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692392756350 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:32:756.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:32:756.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:32:804.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:804.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692392804523 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:804.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:804.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:804.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692392804944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:804.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:32:805.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:32:805.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:32:805.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:32:805.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692392805445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:32:805.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:813.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:813.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692393813558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:813.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:814.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:814.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692393814217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37705 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:814.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:814.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:33:814.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:33:814.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:33:814.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692393814648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:33:814.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:240.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:240.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692394240352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:240.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:240.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:240.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692394240770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:240.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:241.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:34:241.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:34:241.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:241.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692394241211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:241.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:241.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:241.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692394241217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:241.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:241.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:241.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692394241275 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:241.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:241.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:241.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692394241475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:241.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:241.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:241.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692394241493 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:34:241.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:242.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:242.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692394242253 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:242.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:242.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:242.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692394242301 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:242.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:242.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:06:34:287.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:287.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394287449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:287.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:287.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:287.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692394287916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45785 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:287.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:288.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:34:288.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:34:288.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:288.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394288353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:288.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:729.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:729.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692394729609 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:729.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:730.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:730.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692394730500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:730.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:730.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:34:730.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:34:731.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:731.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394730879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:731.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:731.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:731.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394730883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:731.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:731.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:731.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394731118 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:731.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:731.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:731.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394731205 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:06:34:731.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:731.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:731.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394731733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:34:731.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:34:731.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:34:732.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692394731769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:34:732.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:34:732.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871