2025-09-01 Mon 02:13:06:147.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692786146851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33585 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:147.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:147.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:06:147.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:06:147.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:147.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692786147426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:147.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:147.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:147.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692786147431 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:147.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:147.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:147.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692786147488 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:06:147.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:06:147.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:147.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692786147594 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:06:147.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:06:147.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:147.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692786147625 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:06:147.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:06:148.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:148.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692786148274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:148.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:148.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:148.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692786148318 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:06:148.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:06:148.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:06:193.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:193.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692786193448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:193.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:194.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:194.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692786194081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33031 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:194.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:06:194.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:06:194.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:06:194.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:06:194.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692786194474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:06:194.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:121.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:121.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692787121622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:121.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:122.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:122.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692787122096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:122.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:122.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:07:122.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:07:122.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:122.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692787122680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:122.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:236.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:236.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692787235981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:236.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:236.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:236.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692787236535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:236.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:236.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:07:237.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:07:237.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:237.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692787237146 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:237.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:321.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:321.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692787321288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:321.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:321.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:321.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692787321455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:321.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:07:321.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:07:321.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:07:322.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:07:322.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692787322012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:07:322.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:102.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:104.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692789102818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:104.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:104.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:103.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:09:104.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692789103435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38193 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:104.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:104.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:09:104.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:104.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692789104612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:104.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:807.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:808.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692789807924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:808.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:808.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:808.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692789808595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:808.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:808.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:09:808.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:09:809.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:809.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692789808990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:809.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:809.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:809.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692789809000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:809.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:809.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:809.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692789809064 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:09:809.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:09:809.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:809.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692789809297 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:09:809.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:09:809.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:809.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692789809316 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:09:809.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:09:810.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:810.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692789810132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:810.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:810.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:09:810.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:810.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692789810193 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:09:810.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:09:856.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:856.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692789856354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:856.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:856.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:856.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692789856916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39723 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:856.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:09:857.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:09:857.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:09:857.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:09:857.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692789857225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:09:857.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:10:820.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:10:820.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692790820371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:10:820.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:10:820.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:10:821.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692790820968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34067 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:10:821.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:10:821.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:10:821.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:10:821.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:10:821.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692790821245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:10:821.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:017.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:017.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692791017307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:017.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:017.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:017.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692791017484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:017.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:017.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:11:017.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:11:017.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:017.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692791017925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:017.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:017.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:017.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692791017930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:017.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:018.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:018.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692791018010 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:11:018.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:11:018.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:018.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692791018068 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:11:018.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:11:018.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:018.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692791018577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:018.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:018.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:018.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692791018617 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:11:018.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:11:018.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:11:071.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:071.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692791071780 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:071.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:071.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:071.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791071939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:071.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:072.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:11:072.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:11:072.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:072.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692791072211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:072.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:390.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:390.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692791390157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:390.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:390.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:390.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692791390385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:390.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:390.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:11:390.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:11:390.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:391.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:11:391.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791390784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:391.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:391.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:391.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791390791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:391.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:391.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:391.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791390855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:11:391.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:11:391.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:391.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791390903 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:11:391.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:11:391.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:391.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791391439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:391.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:391.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:391.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692791391479 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:11:391.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:11:437.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:437.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692791437645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:437.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:438.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:438.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692791438252 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:438.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:438.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:11:438.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:11:438.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:438.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692791438663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:438.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:026.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:026.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692792026380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:026.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:026.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:026.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692792026566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46289 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:026.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:026.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:12:026.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:12:026.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:026.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692792026863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:026.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:446.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:446.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692792446837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:446.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:447.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:447.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692792447275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:447.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:447.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:12:447.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:12:447.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:447.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692792447720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:447.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:765.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:765.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692792765397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:765.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:765.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:12:765.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:12:766.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:766.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692792765588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35783 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:766.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:12:766.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:12:766.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692792765884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:12:766.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:100.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:100.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692793100773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:100.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:101.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:101.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692793101263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32837 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:101.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:101.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:13:101.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:13:101.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:101.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692793101805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:101.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:101.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:101.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692793101814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:101.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:101.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:101.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692793101919 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:13:101.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:13:102.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:102.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692793102043 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:13:102.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:13:102.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:102.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692793102070 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:13:102.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:13:102.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:102.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692793102906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:102.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:102.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:102.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:13:103.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692793102968 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:13:103.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:13:154.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:154.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692793154097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:154.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:154.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:154.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692793154794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:154.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:155.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:13:155.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:13:155.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:155.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692793155233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:155.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:007.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:007.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692794006969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:007.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:007.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:007.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692794007423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36275 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:007.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:007.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:14:007.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:14:007.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:007.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692794007794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:007.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:109.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:109.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:14:110.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692794109353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:110.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:110.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:110.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:14:110.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692794109541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41567 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:110.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:110.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:110.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692794110776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:110.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:301.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:302.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692794301820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:302.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:302.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:302.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692794301996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45101 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:302.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:302.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:14:302.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:14:302.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:14:302.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692794302380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:14:302.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:000.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:001.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692796000867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:001.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:001.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:001.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692796001055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:001.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:001.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:16:001.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:16:001.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:001.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692796001472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:001.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:726.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:726.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692796726033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:726.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:726.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:726.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692796726655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44611 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:726.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:726.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:16:726.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:16:726.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692796726979 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:727.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:727.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692796726985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:727.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:727.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692796727064 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:16:727.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:16:727.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692796727163 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:16:727.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:16:727.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692796727191 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:16:727.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:16:727.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692796727718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:727.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:727.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:727.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692796727763 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:16:727.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:16:727.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:16:772.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:773.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692796772937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:773.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:773.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:773.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692796773333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36327 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:773.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:16:773.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:16:773.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:16:773.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:16:773.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692796773879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:16:773.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:17:701.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:17:701.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692797701023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:17:701.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:17:701.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:17:701.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692797701441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:17:701.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:17:701.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:17:702.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:17:702.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:17:702.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692797702302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:17:702.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:147.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:148.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798147952 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:148.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:148.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:148.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692798148540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:148.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:148.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:18:148.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:18:148.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:149.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798148868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:149.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:149.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:149.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798148878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:149.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:149.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:149.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798148939 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:18:149.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:18:149.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:149.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798148989 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:18:149.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:18:149.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:149.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798149546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:149.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:149.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:149.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798149585 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:18:149.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:18:149.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:18:198.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:198.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798198711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:198.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:198.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:198.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692798198885 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35627 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:198.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:199.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:18:199.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:18:199.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:199.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798199347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:199.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:370.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:370.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692798370541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:370.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:371.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:371.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692798371124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:371.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:371.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:18:371.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:18:371.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:371.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798371517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:371.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:371.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:371.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798371525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:371.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:371.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:371.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798371608 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:18:371.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:18:371.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:371.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798371659 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:18:371.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:18:372.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:372.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692798372197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:372.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:372.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:372.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:18:372.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692798372239 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:18:372.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:18:420.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:420.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692798420361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:420.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:420.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:420.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692798420822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36099 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:420.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:18:421.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:18:421.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:18:421.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:18:421.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692798421155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:18:421.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:108.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:108.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692799108665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:108.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:109.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:109.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692799109127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:109.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:109.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:19:109.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:19:111.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:111.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692799111868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:111.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:384.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:384.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692799384350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:384.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:384.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:384.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692799384584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:384.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:384.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:19:384.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:19:384.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:384.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692799384934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:384.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:790.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:790.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692799790534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:790.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:790.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:790.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692799790755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:790.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:791.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:19:791.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:19:791.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:791.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692799791281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:791.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:889.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:889.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692799889721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:889.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:890.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:890.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692799890263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:890.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:890.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:19:890.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:19:890.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:890.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692799890788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:890.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:890.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:890.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692799890799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:890.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:890.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:890.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692799890908 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:19:890.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:19:891.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:891.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692799891029 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:19:891.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:19:891.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:891.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692799891062 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:19:891.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:19:891.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:891.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692799891680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:891.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:891.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:891.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692799891721 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:19:891.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:19:891.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:19:936.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:937.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692799936914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:937.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:937.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:937.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692799937543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:937.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:19:937.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:19:937.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:19:937.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:19:937.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692799937949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:19:938.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:20:912.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:20:912.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692800912035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:20:912.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:20:912.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:20:912.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692800912537 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:20:912.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:20:912.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:20:913.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:20:913.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:20:913.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692800913055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:20:913.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:000.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:21:000.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692801000130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:21:000.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:000.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:21:000.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692801000612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:21:000.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:000.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:21:001.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:21:001.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:21:001.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692801001046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:21:001.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:257.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:21:257.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692801257130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:21:257.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:257.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:21:257.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692801257330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:21:257.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:257.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:21:257.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:21:257.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:21:257.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692801257691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:21:257.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:21:770.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:56886 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:21:780.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:56894 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:23:017.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:017.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692803017178 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:017.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:017.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:017.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692803017786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:017.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:018.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:23:018.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:23:018.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:018.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692803018108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:018.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:499.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:499.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692803499433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:499.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:500.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:500.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692803500036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:500.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:500.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:23:500.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:23:500.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:500.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692803500477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:500.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:500.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:500.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692803500484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:500.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:500.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:500.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692803500616 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:23:500.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:23:501.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:501.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692803501412 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:23:501.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:23:501.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:501.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692803501435 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:23:501.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:23:502.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:502.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692803502591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:502.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:502.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:502.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:23:502.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692803502633 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:23:502.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:23:549.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:549.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692803549628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:549.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:550.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:550.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692803550197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:550.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:23:550.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:23:550.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:23:550.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:23:550.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692803550911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:23:550.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:469.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:470.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:24:470.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692804469097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:470.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:470.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:470.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692804469827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:470.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:470.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:24:470.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:470.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692804470888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:470.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:857.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:857.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692804857837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:857.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:858.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:858.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692804858521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:858.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:858.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:24:858.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:24:858.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:858.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692804858810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:858.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:858.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:858.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692804858826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:858.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:858.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:858.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692804858904 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:24:858.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:24:858.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:858.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692804858963 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:24:858.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:24:859.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:859.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692804859603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:859.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:859.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:859.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692804859654 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:24:859.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:24:859.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:24:905.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:906.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692804905886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:906.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:906.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:906.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692804906659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37977 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:906.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:907.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:24:907.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:24:907.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:907.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692804907122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:907.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:981.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:981.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692804981250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:981.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:981.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:981.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692804981429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:981.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:981.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:24:981.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:24:981.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:981.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692804981896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:981.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:981.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:981.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692804981902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:981.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:981.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:982.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692804981975 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:24:982.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:24:982.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:982.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692804982025 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:24:982.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:24:982.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:982.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692804982568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:24:982.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:982.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:982.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692804982611 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:24:982.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:24:982.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:25:034.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:034.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692805034737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:034.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:035.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:035.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692805035306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:035.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:035.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:25:035.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:25:035.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:035.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692805035638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:035.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:807.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:807.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692805807340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:807.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:807.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:807.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692805807712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:807.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:808.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:25:808.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:25:808.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:808.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692805808268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:808.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:980.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:980.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692805980787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:980.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:981.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:981.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692805981232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:981.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:25:981.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:25:981.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:25:981.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:981.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692805981642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:25:981.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:26:559.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:26:559.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692806559051 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:26:559.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:26:559.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:26:559.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692806559262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:26:559.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:26:559.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:26:559.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:26:559.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:26:559.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692806559672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:26:559.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:050.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:050.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807050362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:050.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:050.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:050.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:27:050.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:27:051.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807050536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:051.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:051.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:051.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807050860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:051.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:051.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:051.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807050865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:051.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:051.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:051.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807050913 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:27:051.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:27:051.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:051.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807051004 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:27:051.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:27:051.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:051.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807051021 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:27:051.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:27:051.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:27:052.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:052.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692807051593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:052.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:052.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:052.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692807051635 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:27:052.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:27:100.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:100.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692807100760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:100.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:101.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:101.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692807101127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:101.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:101.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:27:101.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:27:101.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:101.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807101628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:101.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:644.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:644.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692807644411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:644.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:644.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:644.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692807644588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:644.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:644.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:27:644.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:27:644.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:644.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692807644965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:645.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:28:015.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:28:015.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692808015731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:28:015.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:28:015.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:28:015.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692808015905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:28:015.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:28:016.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:28:016.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:28:016.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:28:016.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692808016242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:28:016.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:28:118.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:28:118.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692808118764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:28:118.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:28:119.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:28:119.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692808119215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45421 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:28:119.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:28:119.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:28:119.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:28:119.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:28:119.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692808119771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:28:119.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:29:955.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:29:956.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692809955945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:29:956.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:29:956.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:29:956.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692809956406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:29:956.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:29:956.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:29:956.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:29:956.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:29:957.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692809956978 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:29:957.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:303.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:303.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692810303820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:304.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:304.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:304.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692810304694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35681 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:304.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:304.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:30:305.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:30:305.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:305.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692810305047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:305.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:305.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:305.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692810305052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:305.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:305.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:305.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692810305220 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:30:305.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:30:305.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:305.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692810305338 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:30:305.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:30:305.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:305.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692810305369 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:30:305.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:30:306.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:306.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692810306584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:306.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:306.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:306.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692810306635 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:30:306.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:30:306.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:30:356.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:357.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692810356920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:357.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:357.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:357.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692810357640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45317 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:357.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:30:357.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:30:357.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:30:358.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:30:358.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692810357913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:30:358.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:375.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:375.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692811375082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:375.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:375.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:375.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692811375270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:375.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:375.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:31:375.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:31:375.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:375.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692811375636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:375.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:425.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:425.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692811425113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:425.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:425.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:425.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692811425314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41993 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:425.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:425.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:31:425.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:31:425.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:425.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692811425724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:425.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:425.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:425.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692811425730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:425.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:425.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:425.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692811425788 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:31:425.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:31:425.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:425.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692811425835 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:31:425.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:31:426.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:426.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692811426389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:426.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:426.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:426.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692811426434 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:31:426.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:31:426.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:31:479.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:479.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692811479856 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:479.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:480.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:480.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692811480495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:480.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:480.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:31:480.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:31:480.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:480.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692811480922 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:31:480.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:31:834.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:835.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692811834789 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 7239 dequeues = 7239 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 7321 dequeues = 7321 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692782869200 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692811480930 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:13:31:835.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:31:835.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:835.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692811834910 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1445 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 127 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1643 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 396 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5575 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1593 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3635, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7239, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:31:835.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:31:842.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:842.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692811842642 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:13:31:842.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:13:31:842.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:842.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692811842685 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 2047 in_bytes = 255196 out_tpkts = 2127 out_bytes = 119145 exception_packets = 66 exception_packets_allowed = 53 raw_vhost_stats= [ name = vhost0 in_pkts = 2047 in_bytes = 255196 out_pkts = 2127 out_bytes = 119145 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.66784e-15 total_out_bandwidth_utilization = 1.58673e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1230, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2927, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1939 in_bytes = 111249 out_pkts = 2050 out_bytes = 255322 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:13:31:842.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:31:842.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:31:842.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692811842719 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:13:31:842.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:13:32:228.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:228.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692812228407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:228.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:228.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:228.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692812228850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37209 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:228.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:229.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:32:229.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:32:229.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:229.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692812229329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:229.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:229.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:229.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692812229339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:229.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:229.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:229.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692812229405 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:32:229.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:32:229.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:229.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692812229466 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:32:229.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:32:230.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:230.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692812230099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:230.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:230.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:230.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692812230155 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:32:230.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:32:230.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:32:279.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:279.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692812279315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:279.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:279.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:279.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692812279855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:279.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:280.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:32:280.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:32:280.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:280.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692812280258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:280.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:417.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:417.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692812417035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:417.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:417.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:417.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692812417465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42113 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:417.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:417.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:32:417.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:32:418.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:418.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692812417997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:32:418.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:32:869.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:13:32:869.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:13:32:869.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:32:869.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692812869439 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:32:869.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:32:869.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:32:869.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:13:32:869.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:13:32:869.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692812869730 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1448 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 129 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1646 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 397 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5586 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1596 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3642, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7258, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:32:870.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:32:870.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:13:32:870.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:13:32:870.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:32:870.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:13:32:870.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:870.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692812870225 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1449 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 130 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1646 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 397 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5586 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1596 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3642, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7260, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:32:870.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:32:870.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:870.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692812870250 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:32:870.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:32:870.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:13:32:870.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:32:870.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692812870341 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1449 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 130 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 397 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5586 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1596 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3643, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7260, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:32:870.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:33:207.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:33:207.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692813207397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:33:207.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:33:207.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:33:208.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692813207954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:33:208.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:33:208.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:33:208.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:33:209.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:33:209.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692813208994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:33:209.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:33:211.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:33:211.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692813211696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:33:211.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:33:211.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:33:211.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692813211962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38173 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:33:211.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:33:212.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:33:212.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:33:212.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:33:212.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692813212278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:33:212.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:169.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:169.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814169123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:169.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:169.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:169.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814169310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:169.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:169.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:34:169.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:34:169.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:169.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814169717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:169.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:169.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:169.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814169722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:169.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:169.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:169.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814169786 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:34:169.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:34:170.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:170.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692814170001 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:34:170.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:34:170.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:170.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692814170027 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:34:170.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:34:170.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:170.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814170589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:170.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:170.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:170.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692814170630 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:34:170.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:34:170.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:34:215.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:215.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692814215811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:215.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:216.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:216.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692814216005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:216.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:216.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:34:216.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:34:216.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:216.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692814216424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:216.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:506.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:506.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692814506140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:506.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:506.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:506.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692814506716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39525 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:506.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:34:507.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:34:507.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:34:507.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:34:507.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692814507314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:34:507.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:35:287.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:35:287.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692815287572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:35:287.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:35:288.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:35:288.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692815288181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42331 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:35:288.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:35:288.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:35:288.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:35:288.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:35:288.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692815288703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:35:288.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:35:333.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:35:333.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692815333102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:35:333.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:35:333.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:35:333.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692815333679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33353 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:35:333.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:35:333.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:35:334.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:35:334.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:35:334.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692815334058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:35:334.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:142.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:142.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692817142876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:142.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:143.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:143.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692817143503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37145 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:143.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:143.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:37:143.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:37:143.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:143.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692817143914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:143.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:244.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:244.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692817244430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:244.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:245.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:245.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692817245093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36993 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:245.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:245.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:37:245.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:37:245.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:245.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692817245376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:245.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:245.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:245.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692817245384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:245.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:245.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:245.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692817245469 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:37:245.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:37:245.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:245.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692817245872 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:37:245.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:37:245.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:245.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692817245894 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:37:245.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:37:246.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:246.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692817246502 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:246.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:246.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:246.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:37:246.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692817246564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:37:246.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:37:292.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:292.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692817292680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:292.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:292.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:292.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692817292850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:292.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:37:293.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:37:293.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:37:293.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:37:293.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692817293296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:37:293.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:283.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:283.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692818283480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:283.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:283.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:283.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692818283732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36411 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:283.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:284.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:38:284.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:38:284.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:284.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692818284164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:284.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:467.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:467.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692818467505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:467.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:467.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:467.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692818467690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46495 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:467.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:468.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:38:468.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:38:468.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:468.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692818468149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:468.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:468.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:38:468.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:469.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692818468165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:469.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:469.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:469.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692818468233 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:38:469.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:38:469.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:469.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692818468282 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:38:469.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:38:469.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:469.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692818468783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:469.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:469.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:469.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692818468830 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:38:469.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:38:517.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:518.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692818517953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:518.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:518.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:518.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692818518144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:518.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:518.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:38:518.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:38:518.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:518.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692818518492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:518.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:970.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:970.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692818970229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:970.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:970.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:970.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692818970434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:970.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:970.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:38:970.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:38:970.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:971.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692818970891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:971.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:971.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:971.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692818970898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:971.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:971.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:971.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:38:971.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692818970959 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:38:971.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:38:971.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:971.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692818971008 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:38:971.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:38:971.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:971.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692818971520 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:38:971.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:38:971.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:38:971.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692818971556 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:38:971.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:39:019.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:019.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692819019722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:019.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:020.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:020.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692819020325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:020.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:020.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:39:020.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:39:020.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:020.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692819020746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:020.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:562.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:562.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692819562666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:562.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:563.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:563.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:39:563.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692819563383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:563.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:564.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:39:564.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:564.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692819564060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:564.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:974.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:975.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692819974908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:975.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:975.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:975.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692819975394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35481 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:975.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:39:975.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:39:975.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:39:975.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:39:975.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692819975907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:39:975.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:084.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:084.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692820084369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:084.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:084.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:084.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692820084898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:084.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:085.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:40:085.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:40:085.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:085.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692820085466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:085.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:852.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:852.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692820852058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:852.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:852.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:852.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692820852270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33281 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:852.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:852.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:40:852.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:40:852.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:852.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692820852661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:852.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:852.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:852.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692820852669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:852.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:852.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:852.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692820852845 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:40:852.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:40:853.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:853.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692820853270 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:40:853.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:40:853.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:853.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692820853290 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:40:853.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:40:854.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:854.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692820854145 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:854.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:854.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:854.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692820854205 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:40:854.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:40:854.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:40:903.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:903.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692820903351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:903.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:903.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:903.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692820903831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:903.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:904.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:40:904.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:40:904.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:904.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692820904156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:904.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:688.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:688.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692821688220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:688.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:688.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:688.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692821688812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:688.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:689.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:41:689.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:41:689.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:689.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692821689092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:689.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:776.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:776.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692821776084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:776.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:776.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:776.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692821776620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:776.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:776.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:41:777.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:41:777.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:777.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692821777151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:777.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:876.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:876.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692821876294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:876.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:876.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:876.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692821876877 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:876.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:41:877.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:41:877.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:41:877.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:41:877.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692821877205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:41:877.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:757.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:757.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692823757631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:757.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:758.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:758.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692823758098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33679 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:758.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:758.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:43:758.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:43:758.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:758.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692823758702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:758.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:758.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:758.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692823758712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:758.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:758.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:758.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692823758831 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:43:758.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:43:758.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:758.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692823758961 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:43:759.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:43:759.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:759.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692823758985 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:43:759.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:43:759.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:759.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692823759612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:759.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:759.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:759.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:43:759.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692823759680 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:43:759.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:43:803.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:803.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692823803360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:803.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:803.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:803.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692823803850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:803.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:804.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:43:804.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:43:804.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:804.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692823804285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:804.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:804.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:804.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692823804795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:804.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:805.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:805.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692823805108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:805.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:43:805.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:43:805.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:43:805.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:43:805.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692823805624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:43:805.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:44:714.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:44:714.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692824714771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:44:714.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:44:715.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:44:716.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692824715979 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:44:716.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:44:716.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:44:716.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:44:716.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:44:716.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692824716355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:44:716.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:701.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:701.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692825701374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:701.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:701.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:701.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692825701845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41149 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:701.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:702.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:45:702.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:45:702.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:702.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825702386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:703.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:703.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:703.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825702394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:703.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:703.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:703.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825702475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:45:703.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:45:703.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:703.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825702543 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:45:703.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:45:703.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:703.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825703100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:703.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:703.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:703.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825703137 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:45:703.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:45:703.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:45:748.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:748.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692825748330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:748.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:45:748.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:748.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:748.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692825748535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:748.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:748.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:45:748.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:749.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692825748970 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:749.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:865.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:865.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692825865168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:865.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:865.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:865.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692825865397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:865.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:865.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:45:865.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:45:865.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:865.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692825865876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:865.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:865.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:865.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692825865884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:865.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:866.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:866.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692825866004 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:45:866.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:45:866.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:866.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692825866070 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:45:866.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:45:866.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:866.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692825866634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:866.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:866.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:866.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:45:866.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692825866680 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:45:866.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:45:912.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:912.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825912867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:912.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:913.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:913.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692825913044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:913.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:45:913.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:45:913.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:45:913.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:45:913.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692825913457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:45:913.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:750.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:750.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692826750507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:750.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:750.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:750.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692826750936 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:750.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:751.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:46:751.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:46:751.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:751.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692826751430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:751.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:775.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:775.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692826775170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:775.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:775.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:776.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692826775951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:776.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:776.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:46:776.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:46:776.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:776.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692826776352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:776.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:876.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:876.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692826876629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:876.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:876.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:876.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692826876806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42537 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:876.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:46:877.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:46:877.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:46:877.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:46:877.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692826877281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:46:877.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:404.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:404.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692827404482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:404.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:404.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:404.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692827404685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40121 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:404.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:404.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:47:405.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:47:405.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692827405075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:405.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:405.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692827405081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:405.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:405.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692827405139 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:47:405.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:47:405.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692827405247 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:47:405.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:47:405.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692827405264 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:47:405.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:47:405.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692827405855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:405.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:405.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:405.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692827405905 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:47:405.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:47:405.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:47:459.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:459.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692827459037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:459.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:459.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:459.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692827459495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:459.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:47:459.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:47:459.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:47:460.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:47:460.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692827460017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:47:460.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:423.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:423.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692828423163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:423.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:423.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:423.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692828423765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34235 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:423.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:424.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:48:424.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:48:424.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:424.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692828424222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:424.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:738.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:738.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692828738543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:738.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:739.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:739.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692828739117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:739.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:739.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:48:739.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:48:739.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:739.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692828739636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:739.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:929.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:929.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692828929404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:929.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:930.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:930.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692828929983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:930.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:48:930.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:48:930.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:48:930.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:48:930.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692828930781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:48:930.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:50:242.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:50:242.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692830242325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:50:242.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:50:242.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:50:242.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692830242505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38199 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:50:242.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:50:242.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:50:242.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:50:242.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:50:242.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692830242928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:50:242.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:035.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:035.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692831035562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:035.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:036.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:036.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692831036152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36447 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:036.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:036.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:51:036.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:51:036.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:036.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692831036611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:036.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:036.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:036.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692831036620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:036.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:036.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:036.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692831036681 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:51:036.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:51:036.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:036.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692831036905 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:51:036.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:51:036.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:036.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692831036940 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:51:037.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:51:037.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:037.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692831037564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:037.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:037.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:037.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692831037607 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:51:037.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:51:037.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:51:089.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:089.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692831089763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:089.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:090.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:090.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692831090166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:090.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:090.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:51:090.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:51:090.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:090.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692831090545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:090.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:017.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:017.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692832017689 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:017.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:017.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:017.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692832017881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33067 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:017.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:018.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:52:018.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:52:018.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:018.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692832018265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:018.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:519.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:519.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692832519797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:519.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:520.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:520.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692832520378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:520.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:520.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:52:520.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:52:520.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:521.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:52:521.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832520767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:521.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:521.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:521.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832520775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:521.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:521.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:521.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832520860 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:52:521.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:52:521.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:521.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832520911 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:52:521.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:52:521.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:521.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832521428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:521.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:521.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:521.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832521489 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:52:521.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:52:569.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:569.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692832569600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:569.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:569.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:569.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692832569804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:569.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:52:570.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:52:570.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:52:570.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:52:570.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692832570109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:52:570.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:142.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:143.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692833142918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:143.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:143.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:143.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692833143106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:143.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:143.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:53:143.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:53:143.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:143.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692833143572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:143.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:143.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:143.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692833143579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:143.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:143.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:143.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692833143660 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:53:143.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:143.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:143.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692833143715 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:53:143.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:144.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:144.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692833144232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:144.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:144.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:144.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:53:144.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692833144275 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:53:144.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:190.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:190.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692833190420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:190.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:190.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:191.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692833190959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42427 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:191.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:191.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:53:191.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:53:191.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:191.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692833191440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:191.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:623.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:623.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692833623236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:623.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:623.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:623.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692833623854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:623.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:624.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:53:624.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:53:624.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:624.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692833624240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:624.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:987.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:987.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692833987109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:987.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:987.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:987.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692833987333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42885 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:987.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:987.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:53:987.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:53:987.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:987.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692833987764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:987.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:987.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:987.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692833987771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:987.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:987.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:987.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692833987834 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:53:987.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:988.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:988.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692833988123 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:53:988.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:988.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:988.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692833988144 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:53:988.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:988.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:988.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692833988716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:53:988.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:988.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:53:988.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692833988750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:53:988.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:53:988.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:54:016.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:016.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692834016405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:016.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:016.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:016.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692834016554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37741 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:016.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:016.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:54:016.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:54:016.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:016.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692834016839 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:016.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:038.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:038.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692834038874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:038.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:039.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:039.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692834039397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38927 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:039.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:039.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:54:039.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:54:039.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:039.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692834039759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:039.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:199.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:199.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692834199567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:199.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:199.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:199.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692834199760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:199.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:200.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:54:200.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:54:200.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:200.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692834200187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:54:200.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:002.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:003.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:55:004.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:55:004.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692835002937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:004.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:005.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:005.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692835003120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42417 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:005.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:005.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:005.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692835004900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:005.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:730.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:730.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692835730383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:730.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:731.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:731.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692835730989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:731.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:731.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:55:731.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:55:731.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:731.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692835731341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:731.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:56:306.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:56:306.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692836306359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:56:306.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:56:306.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:56:306.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692836306829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:56:306.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:56:307.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:56:307.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:56:307.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:56:307.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692836307370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:56:307.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:57:003.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:57:003.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692837003062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:57:003.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:57:003.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:57:003.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692837003514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:57:003.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:57:003.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:57:003.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:57:004.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:57:004.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692837004001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:57:004.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:049.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:049.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692838048991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:049.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:049.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:049.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692838049169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:049.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:049.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:58:049.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:58:049.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:049.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692838049486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:049.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:049.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:049.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692838049492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:049.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:049.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:049.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692838049607 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:58:049.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:58:049.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:049.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692838049950 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:58:050.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:58:050.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:050.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692838049970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:58:050.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:58:050.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:050.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692838050519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:050.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:050.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:050.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692838050559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:58:050.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:58:050.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:58:101.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:101.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692838101731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:101.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:102.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:102.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692838102315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45223 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:102.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:58:102.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:58:102.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:58:102.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:58:102.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692838102779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:58:102.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:093.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:093.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692839092922 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:093.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:093.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:093.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692839093120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35583 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:093.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:093.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:59:093.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:59:093.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:093.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692839093642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:093.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:835.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:835.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692839835500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:835.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:836.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:836.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692839836194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:836.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:836.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:59:836.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:59:836.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:836.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692839836510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:836.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:836.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:836.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692839836517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:836.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:836.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:836.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692839836626 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:59:836.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:59:836.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:836.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692839836696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:13:59:836.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:59:837.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:837.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692839837257 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:837.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:837.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:837.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692839837308 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:59:837.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:59:837.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:13:59:890.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:890.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692839890526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:890.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:891.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:891.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692839891024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:891.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:59:891.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:59:891.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:59:891.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:59:891.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692839891558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:59:891.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:267.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:268.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:00:268.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840267538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:268.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:268.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:268.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840267727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40071 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:269.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:269.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:00:269.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:269.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840269299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:269.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:269.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:269.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840269307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:269.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:269.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:269.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840269407 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:00:269.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:269.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:269.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840269495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:00:269.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:270.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:270.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692840270311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:270.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:270.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:270.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692840270364 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:00:270.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:270.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:00:315.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:315.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692840315530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:315.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:315.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:315.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692840315737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35865 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:315.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:315.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:00:316.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:00:316.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:316.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840316102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:316.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:640.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:640.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692840640196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:640.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:640.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:640.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692840640405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:640.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:640.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:00:640.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:00:640.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:640.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692840640828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:640.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:640.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:640.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692840640835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:640.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:640.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:640.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692840640914 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:00:640.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:641.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:641.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692840641005 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:00:641.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:641.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:641.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692840641023 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:00:641.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:641.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:641.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692840641610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:641.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:641.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:641.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:00:641.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692840641651 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:00:641.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:00:691.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:691.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692840691850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:691.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:692.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:692.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692840692317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:692.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:692.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:00:692.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:00:692.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:692.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692840692623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:692.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:944.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:944.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692840944750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:944.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:945.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:945.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692840945328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:945.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:00:945.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:00:945.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:00:945.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:00:945.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692840945851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:00:945.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:181.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:181.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692841181785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:181.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:182.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:182.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692841182248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:182.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:182.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:01:182.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:01:182.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:182.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692841182829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:182.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:270.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:270.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692841270251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:270.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:270.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:270.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692841270436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46309 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:270.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:270.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:01:270.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:01:270.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:270.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692841270761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:270.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:700.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:700.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692841700719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:700.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:701.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:701.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692841701218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40113 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:701.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:701.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:01:701.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:01:701.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:701.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692841701628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:01:701.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:01:835.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:835.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692841835060 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 7564 dequeues = 7564 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 7649 dequeues = 7649 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692812870307 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692841701636 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:14:01:835.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:01:835.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:835.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692841835212 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1510 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1716 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 414 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5827 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1665 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3798, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7564, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:01:835.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:01:842.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:842.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692841842875 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:14:01:842.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:14:01:842.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:843.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692841842923 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1354580704 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.17 five_min_avg = 0.115 fifteen_min_avg = 0.125 ] cpu_share = 1.125 ] ] 2025-09-01 Mon 02:14:01:843.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:01:843.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:843.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692841842929 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.125 used_sys_mem = 2854240503 one_min_cpuload = 0.17 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:14:01:843.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:14:01:843.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:843.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692841842961 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 2126 in_bytes = 260786 out_tpkts = 2206 out_bytes = 123315 exception_packets = 68 exception_packets_allowed = 55 raw_vhost_stats= [ name = vhost0 in_pkts = 2126 in_bytes = 260786 out_pkts = 2206 out_bytes = 123315 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2015 in_bytes = 115293 out_pkts = 2129 out_bytes = 260912 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:14:01:843.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:01:843.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:01:843.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692841843003 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:14:01:843.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:14:02:870.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:14:02:870.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:14:02:870.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:02:870.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692842870534 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:02:870.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:02:870.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:02:871.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692842870810 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1510 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 134 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1716 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 414 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5827 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1665 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3798, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7571, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:02:871.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:14:02:871.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:14:02:871.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:02:871.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:14:02:871.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:14:02:871.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:02:871.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Idle 2025-09-01 Mon 02:14:02:871.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:02:871.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692842871248 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1511 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 135 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1716 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 414 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5827 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1665 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3798, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7573, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:02:871.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:02:871.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:02:871.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692842871276 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:02:871.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:02:871.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:14:02:871.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:02:871.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692842871370 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1511 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 135 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1717 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 414 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5827 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1665 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3799, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7573, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:02:871.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:02:890.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:02:890.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692842890019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:02:890.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:02:890.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:02:890.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692842890590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:02:890.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:02:891.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:02:891.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:02:891.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:02:891.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692842891107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:02:891.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:03:232.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:03:233.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692843232930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:03:233.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:03:233.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:03:233.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692843233541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34507 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:03:233.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:03:233.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:03:233.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:03:233.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:03:233.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692843233810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:03:233.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:03:609.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:03:609.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692843609752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:03:609.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:03:610.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:03:610.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692843610396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41807 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:03:610.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:03:610.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:03:610.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:03:610.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:03:610.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692843610761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:03:610.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:855.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:855.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844855124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:855.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:855.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:855.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844855379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35943 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:855.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:855.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:04:855.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:04:855.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:855.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844855809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:855.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:855.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:855.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844855825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:856.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:856.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:856.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844855888 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:04:856.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:04:856.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:856.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692844856152 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:04:856.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:04:856.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:856.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692844856172 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:04:856.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:04:856.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:856.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844856714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:856.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:04:856.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:856.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:856.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692844856754 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:04:856.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:04:906.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:906.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692844905980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:906.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:906.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:906.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692844906485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43479 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:906.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:04:906.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:04:906.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:04:907.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:04:907.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692844907022 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:04:907.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:05:816.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:05:816.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692845816137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:05:816.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:05:816.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:05:816.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692845816586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42813 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:05:816.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:05:816.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:05:816.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:05:816.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:05:817.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692845816966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:05:817.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:563.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:563.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692846563201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:563.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:563.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:563.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692846563383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39585 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:563.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:563.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:06:563.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:06:563.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:564.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692846563782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:564.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:564.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:564.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692846563787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:564.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:564.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:564.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692846563850 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:06:564.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:06:564.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:564.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692846563909 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:06:564.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:06:564.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:564.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692846564453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:564.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:564.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:564.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692846564491 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:06:564.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:06:564.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:06:617.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:617.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692846617643 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:617.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:618.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:618.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692846618065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42293 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:618.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:06:618.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:06:618.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:06:618.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:06:618.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692846618490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:06:618.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:283.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692847282911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:283.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:283.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692847283095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:283.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:283.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:07:283.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:07:283.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692847283470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:283.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:283.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692847283479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:283.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:283.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692847283544 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:07:283.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:283.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692847283774 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:07:283.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:283.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:283.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692847283792 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:07:283.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:284.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:284.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692847284366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:284.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:284.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:284.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:07:284.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692847284399 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:07:284.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:336.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:336.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692847336546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:336.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:336.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:336.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847336733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:336.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:337.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:07:337.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:07:337.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:337.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692847337235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:337.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:446.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:446.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692847445965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:446.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:446.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:446.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692847446466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33405 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:446.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:446.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:07:446.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:07:446.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:446.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847446855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:446.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:446.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:446.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847446862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:446.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:446.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:447.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847446949 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:07:447.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:447.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:447.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847446998 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:07:447.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:447.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:447.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692847447543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:447.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:447.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:07:447.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:447.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692847447573 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:07:447.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:07:492.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:492.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692847492712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:492.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:493.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:493.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692847493246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:493.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:493.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:07:493.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:07:493.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:493.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847493935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:493.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:635.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:635.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692847635649 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:635.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:636.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:636.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847636227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38713 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:636.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:636.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:07:636.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:07:636.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:636.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692847636682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:636.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:923.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:923.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692847923104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:923.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:923.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:923.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692847923342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:923.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:07:923.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:07:923.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:07:923.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:07:923.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692847923823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:07:923.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:282.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:282.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692848282412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:282.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:282.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:282.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692848282597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:282.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:282.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:08:283.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:08:283.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:283.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692848283249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:283.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:493.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:493.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692848493099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:493.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:493.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:493.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692848493733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:493.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:494.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:08:494.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:08:494.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:494.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692848494133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:494.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:09:670.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:09:671.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692849670772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:09:671.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:09:671.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:09:671.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692849670951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:09:671.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:09:671.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:09:671.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:09:671.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:09:671.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692849671350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:09:671.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:10:101.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:10:101.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692850101401 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:10:101.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:10:101.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:10:101.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692850101684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35927 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:10:101.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:10:102.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:10:102.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:10:102.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:10:102.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692850102105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:10:102.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:10:294.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:10:294.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692850294310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:10:294.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:10:294.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:10:294.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692850294822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43687 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:10:294.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:10:295.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:10:295.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:10:295.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:10:295.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692850295307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:10:295.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:668.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:668.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692851668047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:668.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:668.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:668.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692851668403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44915 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:668.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:668.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:11:668.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:11:668.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:669.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692851668843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:669.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:669.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:669.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692851668850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:669.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:669.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:669.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692851668927 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:11:669.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:11:669.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:669.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692851669172 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:11:669.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:11:669.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:669.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692851669196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:11:669.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:11:670.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:670.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692851670018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:670.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:670.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:670.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692851670060 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:11:670.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:11:670.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:11:716.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:716.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692851716267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:716.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:716.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:716.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692851716793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42425 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:716.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:11:717.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:11:717.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:11:717.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:11:717.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692851717143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:11:717.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:12:743.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:743.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692852743302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:743.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:12:743.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:743.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692852743756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:743.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:12:744.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:12:744.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:12:744.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:744.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692852744159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:744.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:559.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:559.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692853559492 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:559.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:559.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:560.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692853559974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45025 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:560.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:560.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:13:560.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:13:560.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:560.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692853560467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:560.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:560.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:560.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692853560473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:560.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:560.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:560.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692853560540 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:13:560.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:13:560.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:560.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692853560603 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:13:560.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:13:561.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:561.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692853561290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:561.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:561.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:561.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692853561347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:13:561.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:13:561.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:13:611.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:611.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:13:611.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:13:611.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692853611479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:611.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:611.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:611.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692853611638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:611.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:13:612.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:13:612.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692853611907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:13:612.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:291.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:291.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692854291421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:291.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:292.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:292.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692854291994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37531 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:292.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:292.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:14:292.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:14:292.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:292.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854292394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:292.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:292.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:292.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854292400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:292.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:292.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:292.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854292476 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:14:292.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:292.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:292.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854292528 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:14:292.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:293.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:293.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854293008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:293.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:293.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:293.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:14:293.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854293060 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:14:293.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:341.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:341.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692854341240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:341.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:341.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:341.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692854341419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:341.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:341.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:14:341.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:14:341.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:341.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692854341825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:341.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:350.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:350.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692854350229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:350.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:350.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:350.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692854350384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:350.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:350.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:14:350.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:14:350.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:350.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692854350802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:350.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:350.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:350.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692854350809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:350.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:350.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:350.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692854350887 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:14:351.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:351.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:351.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692854350978 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:14:351.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:351.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:351.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692854350996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:14:351.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:351.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:351.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692854351538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:351.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:351.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:351.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692854351592 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:14:351.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:14:351.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:14:403.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:403.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692854403767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:403.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:404.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:404.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692854404121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:404.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:404.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:14:404.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:14:404.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:404.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692854404583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:404.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:575.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:575.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692854575044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:575.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:575.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:575.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692854575919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:575.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:576.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:14:576.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:14:576.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:576.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692854576168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:576.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:832.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:832.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692854832320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:832.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:832.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:832.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692854832772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:832.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:14:833.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:14:833.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:14:833.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:14:833.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692854833143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:14:833.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:15:313.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:15:313.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692855313706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:15:313.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:15:313.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:15:313.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692855313900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42263 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:15:313.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:15:313.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:15:313.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692855313914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:15:313.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:15:313.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:15:313.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692855313982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37279 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:15:314.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:15:314.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:15:314.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:15:314.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:15:314.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:15:314.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692855314297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:15:314.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:15:314.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:15:314.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:15:314.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692855314415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:15:314.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:16:376.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:16:376.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692856376347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:16:376.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:16:376.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:16:376.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692856376529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36817 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:16:376.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:16:376.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:16:376.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:16:376.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:16:376.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692856376841 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:16:376.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:17:204.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:17:205.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692857204472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:17:205.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:17:205.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:17:205.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:17:206.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692857205059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39899 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:17:206.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:17:206.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:17:206.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:17:206.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692857206154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:17:206.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:17:240.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:17:240.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692857240526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:17:240.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:17:240.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:17:240.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692857240932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:17:240.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:17:241.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:17:241.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:17:241.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:17:241.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692857241472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:17:241.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:649.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:649.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692858649307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:649.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:649.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:649.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692858649807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44199 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:649.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:650.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:18:650.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:18:650.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:650.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:650.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:650.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:650.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:651.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:651.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:651.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:18:651.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650242 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:18:651.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:18:651.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:651.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650379 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:18:651.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:18:651.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:651.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650411 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:18:651.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:18:651.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:651.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:651.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:651.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:651.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858650971 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:18:651.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:18:704.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:704.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692858704098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:704.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:704.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:704.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858704600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35947 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:704.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:704.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:18:705.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:18:705.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:18:705.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692858705320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:18:705.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:19:740.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:19:740.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692859740504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:19:740.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:19:740.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:19:740.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692859740724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:19:740.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:19:741.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:19:741.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:19:741.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:19:741.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692859741248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:19:741.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:301.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:301.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692860300982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:301.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:301.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:301.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692860301161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:301.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:301.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:20:301.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:20:302.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:302.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860301650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:302.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:302.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:302.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860301666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:302.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:302.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:302.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860301720 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:20:302.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:302.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:302.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860301770 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:20:302.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:302.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:302.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860302290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:302.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:302.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:302.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860302323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:20:302.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:302.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:20:354.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:354.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692860354504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:354.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:355.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:355.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692860355083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:355.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:355.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:20:355.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:20:355.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:355.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692860355526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:355.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:914.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:914.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692860914356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:914.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:915.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:915.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692860914982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:915.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:915.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:20:915.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:20:915.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:915.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692860915756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:915.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:915.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:915.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692860915771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:915.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:915.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:915.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692860915828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:20:915.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:915.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:915.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692860915928 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:20:915.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:915.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:916.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692860915945 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:20:916.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:916.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:916.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692860916623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:916.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:916.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:916.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692860916672 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:20:916.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:20:916.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:20:968.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:968.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692860968805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:968.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:969.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:969.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692860969430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35043 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:969.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:969.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:20:969.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:20:969.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:20:969.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692860969782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:20:969.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:327.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:327.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692861327705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:328.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:328.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:328.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692861328354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:328.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:328.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:21:328.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:21:328.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:328.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692861328901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:328.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:489.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:489.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692861489636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:489.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:490.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:490.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692861490232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:490.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:490.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:21:491.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:21:491.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:491.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692861491102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:491.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:491.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:491.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692861491109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:491.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:491.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:491.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692861491181 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:21:491.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:21:491.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:491.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692861491250 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:21:491.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:21:492.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:492.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692861491974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:492.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:492.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:492.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692861492113 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:21:492.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:21:492.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:21:544.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:544.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692861544703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:544.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:545.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:545.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692861545425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33361 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:545.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:545.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:21:545.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:21:545.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:545.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692861545764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:545.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:847.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:847.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692861847373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:847.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:847.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:847.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692861847919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:847.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:21:848.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:21:848.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:21:848.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:21:848.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692861848415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:21:848.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:022.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:023.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692862022925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:023.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:023.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:023.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692862023343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:023.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:023.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:22:023.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:22:023.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:023.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692862023658 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:023.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:473.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:473.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:22:474.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692862472980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:475.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:475.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:475.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692862473604 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34591 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:475.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:475.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:22:475.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:475.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692862475130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:475.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:583.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:37856 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:22:594.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:37862 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:23:417.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:23:417.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692863417084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:23:417.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:23:417.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:23:417.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692863417337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:23:417.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:23:417.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:23:417.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:23:417.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:23:417.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692863417849 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:23:417.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:23:967.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:23:967.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692863967894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:23:968.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:23:968.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:23:968.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692863968448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36097 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:23:968.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:23:968.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:23:968.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:23:968.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:23:968.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692863968765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:23:968.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:24:581.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:581.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692864581326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:24:581.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:24:582.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:582.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692864582674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44039 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:24:582.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:24:582.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:24:582.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:24:582.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:582.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692864582968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:24:583.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:592.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:592.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692865592504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:592.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:592.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:592.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692865592950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:592.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:593.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:25:593.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:25:593.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:593.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692865593619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:593.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:593.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:593.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692865593626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:593.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:593.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:593.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692865593708 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:25:593.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:25:593.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:593.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692865593817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:25:593.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:25:593.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:593.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692865593834 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:25:593.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:25:594.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:594.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692865594430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:594.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:594.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:594.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692865594475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:25:594.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:25:594.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:25:640.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:640.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692865640704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:640.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:641.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:641.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692865641085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:641.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:641.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:25:641.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:25:641.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:641.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692865641636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:25:641.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:26:586.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:26:586.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692866586744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:26:586.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:26:587.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:26:587.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692866587407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:26:587.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:26:587.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:26:587.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:26:587.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:26:587.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692866587824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:26:587.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:593.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:595.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692867593943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:595.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:595.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:595.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692867594471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:595.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:596.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:27:596.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:27:596.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:596.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692867596335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:596.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:596.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:596.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692867596341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:596.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:596.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:596.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692867596419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:27:596.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:596.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:596.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692867596469 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:27:596.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:597.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:597.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692867597044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:597.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:597.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:597.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692867597118 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:27:597.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:597.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:27:642.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:642.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692867642265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:642.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:642.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:642.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692867642430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35861 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:642.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:642.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:27:642.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:27:642.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:642.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692867642793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:642.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:748.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:749.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692867748953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:749.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:749.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:749.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692867749507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:749.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:749.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:27:749.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:27:749.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692867749983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:750.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:750.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692867749990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:750.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:750.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692867750047 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:27:750.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:750.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692867750164 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:27:750.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:750.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692867750190 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:27:750.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:750.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692867750739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:750.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:750.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:750.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692867750783 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:27:750.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:27:751.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:27:804.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:804.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692867804173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:804.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:804.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:804.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692867804747 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:804.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:27:805.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:27:805.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:27:805.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:27:805.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692867805241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:27:805.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:399.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:399.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868399092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:399.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:399.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:399.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868399269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45035 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:399.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:399.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:28:399.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:28:399.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:399.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868399552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:399.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:399.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:399.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868399562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:399.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:399.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:399.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868399638 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:28:399.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:28:399.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:399.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868399686 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:28:399.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:28:400.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:400.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:28:400.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868400198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:400.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:400.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:400.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868400245 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:28:400.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:28:447.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:447.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692868447418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:447.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:447.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:447.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868447627 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33023 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:447.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:447.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:28:448.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:28:448.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:448.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692868448071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:448.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:604.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:604.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692868603981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:604.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:604.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:604.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692868604154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33825 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:604.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:604.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:28:604.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:28:604.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:605.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692868604986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:605.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:614.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:614.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692868614884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:614.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:615.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:615.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692868615163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:615.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:615.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:28:615.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:28:615.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:615.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692868615482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:615.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:786.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:786.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692868786418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:786.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:786.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:786.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692868786961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:786.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:28:787.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:28:787.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:28:787.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:28:787.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692868787518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:28:787.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:29:402.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:29:402.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692869402196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:29:402.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:29:402.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:29:402.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692869402388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38389 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:29:402.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:29:402.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:29:402.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:29:402.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:29:402.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692869402895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:29:402.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:30:613.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:30:613.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692870613619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:30:613.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:30:614.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:30:614.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692870614137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:30:614.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:30:614.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:30:614.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:30:614.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:30:614.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692870614554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:30:614.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:30:839.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:30:839.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692870839692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:30:839.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:30:839.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:30:839.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692870839880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44613 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:30:840.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:30:840.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:30:840.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:30:840.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:30:840.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692870840365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:30:840.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:31:473.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:473.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692871473008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:31:473.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:31:473.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:473.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692871473609 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:31:473.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:31:473.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:31:474.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:31:474.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:474.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692871474291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:31:474.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:31:835.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:835.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692871835371 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 7865 dequeues = 7865 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 7953 dequeues = 7953 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692842871333 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692871474296 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:14:31:835.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:31:835.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:835.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692871835500 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1568 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 137 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1786 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 430 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6060 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1732 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3951, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7865, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:31:835.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:31:843.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:843.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692871843152 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:14:31:843.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:14:31:843.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:843.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692871843196 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 2214 in_bytes = 277358 out_tpkts = 2298 out_bytes = 128815 exception_packets = 69 exception_packets_allowed = 56 raw_vhost_stats= [ name = vhost0 in_pkts = 2214 in_bytes = 277358 out_pkts = 2298 out_bytes = 128815 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.80337e-15 total_out_bandwidth_utilization = 1.60137e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1255, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2954, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2104 in_bytes = 120667 out_pkts = 2217 out_bytes = 277484 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:14:31:843.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:31:843.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:31:843.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692871843248 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:14:31:843.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:14:32:493.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:493.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692872493199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:493.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:493.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:493.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872493411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:493.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:493.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:32:493.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:32:493.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:493.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692872493855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:494.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:494.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:494.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692872493861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:494.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:494.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:494.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692872493923 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:32:494.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:32:494.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:494.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872494159 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:32:494.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:32:494.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:494.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872494178 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:32:494.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:32:494.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:494.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872494754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:494.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:494.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:494.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872494789 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:32:494.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:32:495.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:32:541.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:541.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692872541248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:541.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:541.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:541.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692872541420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36303 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:541.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:541.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:32:541.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:32:541.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:541.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692872541798 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:32:541.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:32:871.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:14:32:873.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:14:32:873.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:32:873.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872873143 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:32:873.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:32:873.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:32:873.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:14:32:873.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:14:32:873.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872873473 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1572 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 139 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1788 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 431 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6068 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1734 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3956, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7882, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:32:873.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:32:873.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:14:32:873.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:14:32:874.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:32:874.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Idle 2025-09-01 Mon 02:14:32:874.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:874.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872873992 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1573 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1788 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 431 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6068 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1734 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3956, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7884, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:32:874.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:32:874.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:874.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872874033 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:32:874.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:32:874.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:14:32:874.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:32:874.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692872874174 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1573 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1789 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 431 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6068 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1734 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 3957, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7884, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:32:874.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:33:594.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:33:595.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692873594939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:33:595.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:33:595.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:33:595.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692873595516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43483 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:33:595.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:33:595.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:33:595.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:33:595.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:33:595.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692873595807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:33:595.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:214.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:214.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692874214710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:214.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:215.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:215.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692874215167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41377 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:215.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:215.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:34:215.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:34:215.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:216.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874215740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:216.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:216.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:216.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874215746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:216.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:216.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:216.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874215811 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:34:216.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:216.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:216.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874215866 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:34:216.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:216.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:216.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874216402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:216.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:216.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:216.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874216438 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:34:216.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:216.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:34:265.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:265.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692874265595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:265.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:265.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:265.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692874265781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:265.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:265.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:34:266.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:34:266.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:266.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692874266095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:266.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:620.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:620.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692874620528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:620.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:621.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:621.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692874621130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37815 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:621.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:621.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:34:621.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:34:621.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:621.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692874621586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:621.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:621.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:621.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692874621593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:621.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:621.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:621.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692874621673 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:34:621.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:621.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:621.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692874621765 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:34:621.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:621.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:621.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692874621785 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:34:621.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:622.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:622.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692874622335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:622.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:622.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:622.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:34:622.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692874622376 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:34:622.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:34:673.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:673.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692874673555 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:673.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:674.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:674.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692874674202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:674.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:34:674.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:34:674.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:34:674.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:34:674.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692874674639 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:34:674.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:247.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:247.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692875247286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:247.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:247.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:247.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692875247818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:247.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:248.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:35:248.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:35:248.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:248.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875248175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:248.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:362.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:362.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692875362459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:362.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:362.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:362.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692875362924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34867 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:362.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:363.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:35:363.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:35:363.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:363.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875363449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:363.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:363.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:363.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875363456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:363.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:363.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:363.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875363515 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:35:363.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:35:363.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:363.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875363562 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:35:363.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:35:364.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:364.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875364087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:364.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:364.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:364.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875364131 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:35:364.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:35:364.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:35:410.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:410.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692875410540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:410.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:411.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:411.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692875411070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:411.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:411.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:35:411.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:35:411.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:411.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692875411508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:411.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:718.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:718.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692875718824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:718.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:719.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:719.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692875719021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33433 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:719.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:719.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:35:719.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:35:719.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:719.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692875719502 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:719.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:737.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:738.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692875737953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:738.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:738.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:738.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692875738532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:738.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:35:738.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:35:738.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:35:738.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:35:738.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692875738859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:35:738.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:36:374.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:36:374.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692876374666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:36:374.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:36:375.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:36:375.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692876375191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44741 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:36:375.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:36:375.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:36:375.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:36:375.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:36:375.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692876375808 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:36:375.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:37:174.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:37:174.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692877174332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:37:174.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:37:174.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:37:174.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692877174531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:37:174.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:37:174.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:37:174.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:37:174.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:37:174.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692877174836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:37:174.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:37:843.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:37:843.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692877843679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:37:843.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:37:843.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:37:843.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692877843886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:37:843.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:37:844.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:37:844.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:37:844.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:37:844.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692877844346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:37:844.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:38:392.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:393.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692878391978 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:392.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:38:393.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:38:393.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:393.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:38:393.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692878392171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41275 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:394.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:38:394.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:394.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692878393742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:394.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:411.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692879411024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:411.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:411.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692879411203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33119 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:411.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:411.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:39:411.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:39:411.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692879411708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:411.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:411.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692879411726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:411.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:411.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692879411784 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:39:411.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:39:411.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692879411880 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:39:411.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:39:411.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:411.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692879411899 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:39:411.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:39:412.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:412.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692879412484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:412.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:412.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:412.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692879412521 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:39:412.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:39:412.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:39:460.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:460.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692879460701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:460.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:461.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:461.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692879461267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:461.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:39:461.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:39:461.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:39:461.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:39:461.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692879461757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:39:461.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:532.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:533.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692880532923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:533.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:533.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:533.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692880533379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:533.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:533.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:40:533.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:40:533.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:533.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692880533951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:533.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:027.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:027.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881027012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:027.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:027.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:027.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692881027736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:027.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:028.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:41:028.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:41:028.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:028.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692881028307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:028.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:028.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:028.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692881028317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:028.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:028.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:028.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692881028376 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:41:028.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:028.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:028.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692881028425 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:41:028.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:029.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:029.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692881028987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:029.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:029.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:029.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692881029026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:41:029.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:029.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:41:074.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:074.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692881074559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:074.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:075.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:075.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692881075147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:075.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:075.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:41:075.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:41:075.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:075.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692881075591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:075.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:624.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:624.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881624547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:624.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:625.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:625.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881625249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:625.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:625.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:41:625.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:41:625.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:625.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881625737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:625.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:625.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:625.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881625745 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:625.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:625.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:625.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881625814 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:41:625.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:625.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:625.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881625951 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:41:626.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:626.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:626.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881625978 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:41:626.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:626.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:626.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692881626751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:626.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:626.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:626.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692881626819 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:41:626.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:41:626.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:41:674.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:674.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881674087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:674.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:674.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:674.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692881674382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:674.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:674.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:41:674.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:41:674.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:674.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692881674815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:674.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:993.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:993.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692881993787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:993.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:994.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:994.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692881994046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39567 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:994.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:41:994.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:41:994.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:41:994.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:41:994.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692881994504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:41:994.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:353.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:354.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692882353894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:354.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:354.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:354.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692882354543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:354.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:354.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:42:354.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:42:354.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:354.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692882354959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:355.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:355.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:355.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692882354966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:355.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:355.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:355.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692882355023 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:42:355.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:42:355.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:355.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692882355090 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:42:355.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:42:355.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:355.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692882355614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:355.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:355.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:355.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692882355654 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:42:355.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:42:355.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:42:407.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:408.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692882407921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:408.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:408.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:408.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692882408094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:408.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:408.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:42:408.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:42:408.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:408.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692882408404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:408.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:586.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:586.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692882586085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:586.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:586.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:586.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692882586664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:586.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:587.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:42:587.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:42:587.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:587.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692882587362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:587.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:710.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:710.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692882710014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:710.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:710.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:710.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692882710614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33719 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:710.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:42:710.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:42:710.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:42:710.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:42:710.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692882710935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:42:710.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:43:398.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:43:398.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692883398554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:43:399.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:43:399.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:43:399.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:43:400.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692883398831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:43:400.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:43:400.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:43:400.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:43:400.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692883400185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:43:400.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:082.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:44:082.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692884082723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:44:083.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:083.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:44:083.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692884082943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:44:083.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:083.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:44:083.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:44:083.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:44:083.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692884083441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:44:083.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:511.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:44:511.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692884511081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:44:511.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:511.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:44:511.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692884511681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:44:511.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:511.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:44:511.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:44:512.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:44:512.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692884512028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:44:512.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:45:200.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:45:200.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692885200370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:45:200.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:45:200.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:45:200.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692885200548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43687 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:45:200.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:45:200.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:45:200.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:45:200.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:45:200.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692885200855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:45:200.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:259.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:259.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692886259530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:259.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:259.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:259.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692886259718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37273 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:259.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:260.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:46:260.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:46:260.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:260.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692886260178 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:260.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:260.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:260.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692886260183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:260.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:260.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:260.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692886260263 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:46:260.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:46:260.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:260.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692886260357 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:46:260.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:46:260.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:260.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692886260383 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:46:260.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:46:260.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:260.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692886260924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:260.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:260.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:46:261.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:261.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692886260966 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:46:261.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:46:311.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:311.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692886311123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:311.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:311.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:311.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692886311310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:311.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:46:311.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:46:311.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:46:311.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:46:311.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692886311740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:46:311.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:283.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:284.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692887283893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:284.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:284.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:284.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692887284088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44501 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:284.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:284.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:47:284.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:47:284.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:284.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692887284488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:284.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:863.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:863.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692887863628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:863.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:864.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:864.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692887864196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44999 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:864.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:864.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:47:864.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:47:864.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:865.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:47:865.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887864678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:865.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:865.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:865.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887864685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:865.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:865.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:865.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887864745 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:47:865.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:47:865.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:865.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887864801 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:47:865.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:47:865.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:865.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887865330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:865.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:865.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:865.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887865370 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:47:865.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:47:916.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:916.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692887916550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:916.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:917.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:917.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692887917111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37827 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:917.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:47:917.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:47:917.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:47:917.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:47:917.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692887917539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:47:917.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:472.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:472.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:48:472.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888472250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:474.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:474.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:48:474.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:474.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888472465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44047 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:474.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:474.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:474.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888474199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:474.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:474.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:474.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888474210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:474.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:474.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:474.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692888474321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:48:474.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:48:474.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:474.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692888474444 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:48:474.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:48:474.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:474.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692888474462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:48:474.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:48:475.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:475.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888475014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:475.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:475.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:475.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888475055 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:48:475.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:48:475.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:48:521.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:521.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692888521197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:521.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:521.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:521.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692888521386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:521.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:521.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:48:521.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:48:521.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:521.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692888521812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:521.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:961.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:961.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692888961704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:961.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:962.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:962.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692888962344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:962.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:48:962.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:48:962.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:48:962.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:48:962.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692888962804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:48:962.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:089.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:089.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692889088996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:089.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:089.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:089.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692889089598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:089.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:089.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:49:089.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:49:089.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:090.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:49:090.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692889089881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:090.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:090.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:090.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692889089888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:090.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:090.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:090.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692889090003 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:49:090.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:49:090.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:090.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692889090066 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:49:090.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:49:090.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:090.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692889090614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:090.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:090.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:090.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692889090652 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:49:090.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:49:142.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:142.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692889142782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:142.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:143.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:143.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692889143249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:143.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:143.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:49:143.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:49:143.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:143.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692889143756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:143.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:228.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:228.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692889228630 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:228.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:228.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:228.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692889228853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34773 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:228.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:229.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:49:229.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:49:229.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:229.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692889229217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:229.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:539.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:539.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692889538967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:539.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:539.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:539.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692889539151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37483 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:539.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:49:539.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:49:539.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:49:539.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:49:539.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692889539648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:49:539.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:50:133.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:50:133.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692890133860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:50:133.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:50:134.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:50:134.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692890134316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36091 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:50:134.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:50:134.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:50:134.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:50:134.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:50:134.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692890134799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:50:134.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:51:015.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:51:015.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692891014971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:51:015.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:51:015.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:51:015.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692891015596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43329 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:51:015.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:51:015.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:51:015.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:51:016.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:51:016.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692891015997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:51:016.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:51:429.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:51:429.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692891429758 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:51:429.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:51:430.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:51:430.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692891430254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:51:430.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:51:430.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:51:430.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:51:430.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:51:430.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692891430796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:51:430.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:52:097.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:52:097.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692892096926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:52:097.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:52:097.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:52:097.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692892097086 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:52:097.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:52:097.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:52:097.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:52:097.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:52:097.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692892097594 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:52:097.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:405.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:405.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692893405371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:405.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:406.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:406.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692893405976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:406.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:406.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:53:406.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:53:406.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:406.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692893406384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:406.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:406.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:406.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692893406392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:406.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:406.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:406.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692893406455 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:53:406.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:53:406.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:406.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692893406707 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:53:406.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:53:406.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:406.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692893406727 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:53:406.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:53:407.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:407.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692893407322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:407.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:407.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:407.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692893407388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:53:407.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:53:407.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:53:455.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:455.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692893455504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:456.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:53:456.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:456.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:53:456.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:456.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692893455702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45007 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:456.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:456.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:456.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692893456196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:456.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:54:509.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:509.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692894509422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:509.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:54:509.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:509.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692894509774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:509.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:54:509.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:54:509.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:54:510.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:510.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692894510030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:510.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:228.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:228.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692895228064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:228.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:228.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:228.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692895228403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:228.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:228.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:55:228.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:55:228.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:229.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895228904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:229.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:229.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:229.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895228910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:229.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:229.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:229.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895228989 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:229.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:229.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:229.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895229052 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:55:229.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:229.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:229.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692895229861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:229.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:229.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:55:229.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:230.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692895229905 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:230.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:275.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:275.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692895275057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:275.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:275.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:275.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692895275627 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:275.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:275.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:55:275.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:55:275.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:275.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692895275921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:275.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:714.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692895714957 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:715.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:715.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692895715138 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:715.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:715.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:55:715.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:55:715.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692895715596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:715.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:715.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692895715612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:715.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:715.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895715719 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:715.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:715.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895715806 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:715.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:715.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:715.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895715824 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:55:715.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:716.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:716.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692895716394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:716.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:716.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:716.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:55:716.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692895716429 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:716.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:765.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:765.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692895765621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:765.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:766.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:766.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692895766223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:766.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:766.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:55:766.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:55:766.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:766.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692895766727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:55:766.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:021.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:021.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692896021783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:021.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:022.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:022.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692896022409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:022.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:022.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:56:022.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:56:022.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:022.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896022825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:022.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:022.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:022.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896022832 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:022.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:022.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:022.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692896022936 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:56:022.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:56:023.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:023.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692896022990 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:14:56:023.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:56:023.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:023.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692896023505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:023.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:023.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:023.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692896023535 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:56:023.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:56:023.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:14:56:072.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:072.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692896072714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:072.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:073.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:073.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692896073279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:073.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:073.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:56:073.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:56:073.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:073.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896073696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:073.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:230.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:230.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896230019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:230.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:230.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:230.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692896230190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37507 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:230.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:230.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:56:230.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:56:230.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:230.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896230578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:230.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:418.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:418.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692896418183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:418.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:418.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:418.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896418460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:418.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:418.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:56:418.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:56:418.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:418.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692896418812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:418.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:684.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:685.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692896684938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:685.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:685.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:685.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692896685405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:685.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:56:685.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:56:685.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:56:685.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:685.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692896685826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:56:685.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:57:009.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:57:009.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692897009876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:57:009.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:57:010.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:57:010.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692897010477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41063 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:57:010.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:57:010.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:57:010.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:57:010.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:57:010.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692897010839 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:57:010.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:084.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:084.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692898084772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:084.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:085.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:085.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692898085390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:085.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:085.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:58:085.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:58:085.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:085.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692898085779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:085.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:737.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:738.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692898737938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:738.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:738.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:738.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692898738518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:738.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:738.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:58:738.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:58:739.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:739.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692898739006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:739.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:883.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:883.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:58:884.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692898883043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:884.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:884.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:884.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692898883428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:884.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:58:884.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:58:884.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:58:884.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692898884777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:58:884.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:487.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:487.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692900487018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:487.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:487.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:487.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692900487656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32783 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:487.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:488.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:00:488.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:00:488.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:488.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692900488180 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:488.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:488.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:488.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692900488187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:488.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:488.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:488.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692900488280 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:00:488.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:00:488.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:488.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692900488600 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:00:488.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:00:488.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:488.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692900488626 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:00:488.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:00:489.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:489.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692900489407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:489.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:489.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:489.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692900489464 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:00:489.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:00:489.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:00:538.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:538.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692900538598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:538.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:539.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:539.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692900539216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:539.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:00:539.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:00:539.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:00:539.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:00:539.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692900539767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:00:539.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:01:475.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:476.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692901475960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:01:476.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:01:476.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:476.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692901476176 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:01:476.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:01:476.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:01:476.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:01:476.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:476.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692901476692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:01:476.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:01:835.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:835.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692901835676 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 8170 dequeues = 8170 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 8261 dequeues = 8261 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692872874105 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692901476721 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:15:01:836.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:01:836.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:836.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692901835819 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1630 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 142 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1854 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 447 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6295 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1799 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4103, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8170, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:01:836.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:01:843.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:843.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692901843413 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:15:01:843.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:15:01:843.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:843.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692901843458 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1367175904 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.22 five_min_avg = 0.145 fifteen_min_avg = 0.135 ] cpu_share = 1.1 ] ] 2025-09-01 Mon 02:15:01:843.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:01:843.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:843.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692901843464 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.1 used_sys_mem = 2854240503 one_min_cpuload = 0.22 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:15:01:843.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:15:01:843.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:843.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692901843490 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 2288 in_bytes = 282610 out_tpkts = 2372 out_bytes = 132727 exception_packets = 71 exception_packets_allowed = 58 raw_vhost_stats= [ name = vhost0 in_pkts = 2288 in_bytes = 282610 out_pkts = 2372 out_bytes = 132727 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2175 in_bytes = 124453 out_pkts = 2291 out_bytes = 282736 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:01:843.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:01:843.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:01:843.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692901843515 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:01:843.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:02:189.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:190.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692902189953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:190.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:190.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:190.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902190427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:190.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:190.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:02:190.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:190.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:190.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692902190865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:190.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:190.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:190.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692902190872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:190.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:191.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:191.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902191011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:191.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:191.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:191.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902191081 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:02:191.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:191.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:191.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902191625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:191.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:191.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:191.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:02:191.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902191667 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:191.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:237.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:237.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902237754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:237.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:237.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:237.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692902237941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:237.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:238.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:02:238.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:238.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:238.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902238408 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:238.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:447.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:447.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902447157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:447.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:447.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:447.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902447691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:447.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:448.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:02:448.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:448.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:448.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902448215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:448.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:448.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:448.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902448222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:448.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:448.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:448.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902448291 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:448.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:448.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:448.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902448399 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:448.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:448.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:448.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902448428 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:02:448.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:449.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:449.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692902449000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:449.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:449.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:449.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692902449045 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:449.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:449.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:02:495.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:495.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902495271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:495.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:495.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:495.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902495440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:495.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:495.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:02:495.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:495.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:495.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902495869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:495.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:628.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:629.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692902628940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:629.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:629.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:629.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692902629149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:629.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:629.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:02:629.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:629.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:629.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902629554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:629.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:629.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:629.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902629562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:629.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:629.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:629.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902629641 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:629.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:629.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:629.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902629698 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:02:629.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:630.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:630.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902630218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:630.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:630.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:630.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692902630274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:630.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:630.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:02:679.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:679.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692902679434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:679.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:679.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:679.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692902679860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42435 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:679.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:680.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:02:680.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:680.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:680.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902680282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:02:680.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:02:874.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:02:874.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:15:02:874.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:02:874.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902874374 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:874.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:874.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:02:874.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:02:874.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:02:874.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902874677 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1640 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 144 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1860 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 450 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6319 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1805 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4118, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8211, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:02:874.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:02:874.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:02:875.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:02:875.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:02:875.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:15:02:875.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:875.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902875058 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1641 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 145 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1860 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 450 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6319 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1805 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4118, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8213, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:02:875.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:02:875.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:875.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902875082 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:02:875.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:02:875.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:02:875.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:02:875.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692902875178 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1641 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 145 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1861 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 450 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6319 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1805 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4119, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8213, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:02:875.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:03:264.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:264.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692903264545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:264.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:264.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:264.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692903264701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:264.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:265.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:03:265.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:03:265.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:265.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903265189 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:265.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:513.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:513.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903513034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:513.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:513.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:513.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903513218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:513.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:513.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:03:513.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:03:513.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:513.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692903513652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:513.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:589.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:589.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692903589356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:589.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:589.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:589.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692903589595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:589.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:589.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:03:589.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:03:589.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:589.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903589942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:589.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:618.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:618.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903618864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:619.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:619.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:619.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903619089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40051 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:619.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:03:619.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:03:619.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:03:619.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:03:619.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692903619512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:03:619.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:317.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:317.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692905317350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:317.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:317.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:317.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692905317836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:317.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:318.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:05:318.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:05:318.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:318.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692905318333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:318.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:511.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:511.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692905511802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:511.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:512.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:512.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692905512445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33513 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:512.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:512.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:05:512.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:05:512.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:512.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692905512791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:512.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:678.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:678.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692905678117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:678.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:678.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:678.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692905678597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:678.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:05:678.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:05:679.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:05:679.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:05:679.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692905679042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:05:679.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:795.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:795.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692907795666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:795.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:796.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:796.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692907796166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40877 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:796.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:796.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:07:796.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:07:796.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:796.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692907796710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:796.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:796.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:796.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692907796718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:796.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:796.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:796.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692907796798 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:07:796.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:796.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:796.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692907796891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:07:796.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:796.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:796.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692907796910 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:07:796.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:797.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:797.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692907797510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:797.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:797.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:07:797.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:797.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692907797674 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:07:797.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:07:845.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:845.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692907845727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:845.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:846.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:846.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692907846337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:846.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:07:846.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:07:846.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:07:846.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:07:846.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692907846678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:07:846.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:755.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:755.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692908755831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:755.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:756.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:756.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692908756339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44203 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:756.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:08:756.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:08:756.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:08:756.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:08:756.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692908756800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:08:756.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:221.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:221.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692909220969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:221.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:221.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:221.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692909221159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:221.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:221.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:221.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:221.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909221644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:222.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:222.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909221659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:222.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:222.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909221717 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:222.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:222.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909221833 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:222.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:222.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909221850 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:09:222.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:222.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909222412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:222.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:222.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:222.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:09:222.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909222444 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:222.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:271.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:271.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692909271632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:271.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:271.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:271.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692909271875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:271.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:272.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:272.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:272.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:272.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692909272390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:272.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:386.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:386.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692909386501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:386.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:387.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:387.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909387091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:387.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:387.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:388.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:388.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:388.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909387673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:388.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:388.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:388.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909387677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:388.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:388.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:388.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909387738 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:388.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:388.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:388.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909387794 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:09:388.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:388.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:388.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909388312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:388.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:388.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:388.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:09:388.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909388351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:388.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:433.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:433.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909433501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:433.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:434.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:434.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692909433992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39391 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:434.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:434.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:434.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:434.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:434.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692909434481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:434.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:819.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:819.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692909819193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:819.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:819.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:819.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692909819683 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38469 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:819.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:819.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:820.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:820.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:820.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692909820091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:820.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:820.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:820.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692909820100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:820.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:820.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:820.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692909820170 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:820.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:820.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:820.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692909820219 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:09:820.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:820.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:820.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909820882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:820.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:820.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:820.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:09:820.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909820936 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:09:821.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:09:866.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:866.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692909866072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:866.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:866.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:866.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692909866493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39575 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:866.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:09:866.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:09:866.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:09:866.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:09:866.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692909866947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:09:867.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:262.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:262.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692910262546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:263.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:263.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:263.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:263.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:263.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692910262736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:263.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:263.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:263.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692910263201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:263.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:370.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:371.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:372.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692910370659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:372.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:372.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:372.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692910370844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37381 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:372.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:372.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:372.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:372.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692910372233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:372.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:791.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:791.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692910791001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:791.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:791.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:791.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692910791602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:791.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:791.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:792.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:792.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:792.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692910792204 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:792.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:857.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:857.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692910857133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:857.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:857.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:857.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692910857359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:857.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:857.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:857.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:857.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:857.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692910857865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:857.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:297.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:297.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692912297390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:297.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:298.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:297.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:12:298.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692912297646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:298.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:298.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:12:298.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:298.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692912298518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:298.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:352.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:352.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692912352344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:352.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:352.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:352.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692912352566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:352.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:352.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:12:353.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:12:353.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:353.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692912353085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:353.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:748.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:748.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692912748067 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:748.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:748.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:748.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692912748322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:748.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:748.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:12:748.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:12:748.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:12:748.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692912748825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:12:748.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:572.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:572.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692914572406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:572.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:572.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:572.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692914572678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39447 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:572.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:572.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:14:573.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:14:573.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:573.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692914573066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:573.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:573.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:573.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692914573083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:573.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:573.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:573.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692914573159 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:14:573.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:573.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:573.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692914573418 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:14:573.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:573.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:573.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692914573441 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:14:573.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:574.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:574.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692914574091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:574.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:574.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:574.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692914574139 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:14:574.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:14:574.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:14:623.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:623.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:14:623.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:14:623.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692914623392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:623.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:623.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:624.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692914623629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:624.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:14:624.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:14:624.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692914623966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:14:624.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:605.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:605.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692915605208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:605.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:605.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:605.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692915605480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35041 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:605.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:605.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:15:605.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:15:605.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:605.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692915605845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:605.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:953.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:953.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692915953271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:953.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:953.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:954.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692915953960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39221 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:954.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:954.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:15:954.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:15:954.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:954.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692915954342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:954.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:954.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:954.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692915954350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:954.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:954.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:954.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692915954451 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:15:954.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:15:954.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:954.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692915954513 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:15:954.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:15:955.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:955.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692915955094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:15:955.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:15:955.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:15:955.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:15:955.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692915955139 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:15:955.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:003.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:003.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916003326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:003.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:003.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:004.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692916003947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:004.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:004.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:004.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:004.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:004.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692916004264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:004.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:042.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:042.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692916042736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:042.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:043.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:043.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692916043031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:043.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:043.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:043.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:043.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:043.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916043477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:043.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:043.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:043.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916043489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:043.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:043.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:043.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916043566 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:16:043.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:043.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:043.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692916043898 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:16:043.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:043.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:043.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692916043924 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:16:043.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:044.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:044.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916044586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:044.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:044.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:044.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:16:044.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916044630 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:16:044.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:097.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:097.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692916097813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:097.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:098.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:098.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916098413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43435 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:098.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:098.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:098.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:098.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:098.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692916098947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:099.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:925.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:925.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692916925049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:925.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:925.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:925.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692916925314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:925.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:925.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:925.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:925.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:925.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692916925773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:925.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:925.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:925.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692916925790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:925.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:925.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:926.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916925959 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:16:926.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:926.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:926.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916926020 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:16:926.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:926.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:926.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916926625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:926.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:926.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:926.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692916926667 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:16:926.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:16:926.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:16:949.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:949.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692916949495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:949.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:949.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:949.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692916949756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:949.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:949.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:950.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:950.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:950.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692916950116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:950.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:977.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:978.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692916977905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:978.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:978.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:978.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692916978151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39425 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:978.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:16:978.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:16:978.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:16:978.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:16:978.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692916978656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:16:978.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:152.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:152.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692917152184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:152.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:152.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:152.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692917152435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44677 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:152.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:17:152.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:152.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:17:152.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:153.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692917152970 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:153.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:676.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:676.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:17:676.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692917675991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:676.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:17:676.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:676.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:676.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692917676253 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:676.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:17:676.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:17:676.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692917676601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:17:676.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:031.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:032.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692918031897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:032.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:032.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:032.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692918032567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:032.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:032.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:18:033.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:18:033.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:033.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692918033147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:033.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:858.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:858.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692918858300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:858.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:858.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:858.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692918858911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36127 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:858.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:18:859.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:18:859.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:18:859.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:18:859.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692918859255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:18:859.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:19:043.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:19:043.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692919043127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:19:043.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:19:043.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:19:043.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692919043360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:19:043.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:19:043.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:19:043.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:19:043.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:19:043.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692919043853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:19:043.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:121.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:121.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692920121353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:121.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:121.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:121.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692920121633 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:121.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:20:121.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:20:122.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:20:122.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:20:122.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692920122074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:20:122.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:492.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:492.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692921492760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:492.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:493.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:493.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692921493367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:493.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:493.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:21:493.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:21:493.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:493.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692921493736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:493.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:493.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:493.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692921493744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:493.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:493.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:494.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692921493805 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:21:494.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:494.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:494.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692921494067 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:21:494.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:494.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:494.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692921494094 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:21:494.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:494.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:494.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692921494663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:494.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:494.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:494.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692921494704 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:21:494.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:21:494.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:21:543.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:543.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692921543828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:543.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:544.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:544.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692921544427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40475 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:544.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:21:544.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:21:544.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:21:544.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:21:544.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692921544878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:21:544.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:508.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:508.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692922508087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:508.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:508.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:508.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692922508385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:508.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:508.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:22:508.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:22:508.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:508.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692922508844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:508.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:963.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:963.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:22:963.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:22:963.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692922963445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:964.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:964.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:964.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692922963645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:964.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:964.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:964.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692922963956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:964.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:964.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:964.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692922963959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:964.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:964.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:964.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692922964168 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:22:964.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:964.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:964.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692922964239 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:22:964.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:964.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:964.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692922964784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:22:964.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:22:965.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:22:965.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692922965150 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:22:965.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:22:965.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:23:015.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:015.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692923014974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:015.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:015.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:015.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692923015606 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39799 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:015.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:015.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:23:015.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:23:016.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:016.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692923016014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:016.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:184.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:184.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692923183997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:184.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:184.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:184.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692923184548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42867 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:184.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:184.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:23:184.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:23:184.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692923184978 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:185.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:185.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692923184986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:185.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:185.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692923185074 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:23:185.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:23:185.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692923185198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:23:185.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:23:185.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692923185216 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:23:185.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:23:185.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692923185787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:185.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:23:185.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:185.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:185.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692923185827 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:23:185.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:23:236.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:236.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692923235993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:236.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:236.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:236.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692923236546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36591 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:236.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:236.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:23:236.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:23:236.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:23:236.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692923236836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:23:236.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:23:412.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:33820 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:23:423.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:33826 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:24:060.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:060.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:060.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692924060191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:060.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:060.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:060.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692924060375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37653 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:060.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:061.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:061.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:061.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692924061163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:061.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:191.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:191.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692924191021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:191.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:191.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:191.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692924191646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:191.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:191.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:191.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:191.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:192.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692924191974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:192.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:298.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:298.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692924298278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:298.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:298.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:298.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692924298875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39051 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:298.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:299.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:299.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:299.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:299.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692924299222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:299.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:299.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:299.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692924299230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:299.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:299.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:299.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692924299327 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:24:299.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:24:299.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:299.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692924299416 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:24:299.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:24:299.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:299.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692924299940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:300.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:300.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:300.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:24:300.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692924300000 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:24:300.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:24:348.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:348.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692924348194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:348.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:348.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:348.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692924348793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39453 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:348.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:348.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:349.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:349.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:349.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692924349093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:349.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:543.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:543.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692924543002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:543.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:543.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:543.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692924543656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:543.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:543.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:543.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:544.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:544.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692924544017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:544.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:366.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:366.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692925366265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:366.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:366.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:366.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692925366872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42155 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:366.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:367.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:25:367.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:25:367.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:367.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692925367284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:367.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:897.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:897.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692925897325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:897.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:897.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:897.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692925897782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46699 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:897.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:898.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:25:898.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:25:898.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:898.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692925898266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:25:898.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:26:028.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:028.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692926028128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:26:028.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:26:028.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:028.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692926028353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39471 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:26:028.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:26:028.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:26:028.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:26:028.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:028.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692926028693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:26:028.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:437.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:437.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692927437450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:437.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:438.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:438.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692927438066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46293 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:438.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:27:438.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:27:438.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:27:438.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:27:438.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692927438353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:27:438.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:540.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:540.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692928540170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:540.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:540.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:540.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692928540782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39223 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:540.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:541.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:541.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:541.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:541.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692928541197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:541.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:541.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:541.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692928541206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:541.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:541.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:541.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692928541317 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:541.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:541.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:541.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692928541419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:541.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:541.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:541.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692928541438 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:28:541.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:542.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:542.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692928542413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:542.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:542.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:542.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:28:542.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692928542458 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:28:542.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:28:590.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:590.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692928590596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:590.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:591.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:591.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692928591036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34373 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:591.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:28:591.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:28:591.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:28:591.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:28:591.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692928591480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:28:591.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:518.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:518.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692929518637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:518.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:519.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:519.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692929519008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:519.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:519.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:29:519.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:29:519.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:519.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692929519434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:519.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:714.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:714.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692929714410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:714.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:714.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:714.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692929714860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:714.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:715.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:29:715.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:29:715.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:715.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692929715334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:715.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:715.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:715.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692929715342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:715.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:715.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:715.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692929715486 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:29:715.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:29:715.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:715.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692929715549 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:29:715.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:29:716.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:716.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692929716107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:716.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:716.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:716.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:29:716.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692929716152 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:29:716.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:29:764.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:764.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692929764322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:764.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:764.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:764.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692929764497 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34551 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:764.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:29:764.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:29:764.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:29:764.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:29:764.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692929764917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:29:764.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:276.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:276.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692930276832 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:277.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:277.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:277.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692930277111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37459 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:277.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:277.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:30:277.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:30:277.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:277.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692930277435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:277.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:277.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:277.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692930277442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:277.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:277.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:277.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692930277545 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:30:277.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:30:277.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:277.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692930277650 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:30:277.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:30:277.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:277.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692930277666 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:30:277.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:30:278.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:278.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692930278303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:278.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:278.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:278.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692930278348 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:30:278.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:30:278.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:30:327.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:327.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692930327637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:327.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:328.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:328.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692930328057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:328.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:328.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:30:328.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:30:328.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:328.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692930328632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:328.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:737.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:737.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692930737103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:737.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:737.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:737.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692930737299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45649 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:737.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:30:737.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:30:737.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:30:737.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:30:737.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692930737730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:30:737.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:318.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:319.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692931318823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:319.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:319.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:319.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692931318976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33247 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:319.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:319.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:319.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:319.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:319.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692931319436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:319.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:506.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:506.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692931506474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:506.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:507.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:507.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692931507131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:507.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:507.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:507.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:507.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:507.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692931507436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:507.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:507.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:507.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692931507441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:507.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:507.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:508.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692931507587 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:31:509.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:31:509.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:509.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692931507641 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:31:509.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:31:509.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:509.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692931509739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:509.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:509.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:509.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692931509779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:31:509.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:31:509.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:31:561.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:561.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692931561027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:561.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:561.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:561.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692931561289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:561.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:561.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:561.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:561.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:561.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692931561702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:561.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:589.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:589.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692931589587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:589.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:590.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:590.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692931590479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:590.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:590.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:31:590.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:31:590.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:590.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692931590809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:31:590.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:31:836.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:836.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692931835969 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 8505 dequeues = 8505 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 8599 dequeues = 8599 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692902875143 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692931590853 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:15:31:836.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:31:836.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:836.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692931836109 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1698 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 147 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1930 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 466 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6552 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1872 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4271, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8505, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:31:836.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:31:843.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:843.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692931843630 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:15:31:843.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:15:31:843.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:843.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692931843674 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 2382 in_bytes = 299525 out_tpkts = 2470 out_bytes = 138515 exception_packets = 74 exception_packets_allowed = 61 raw_vhost_stats= [ name = vhost0 in_pkts = 2382 in_bytes = 299525 out_pkts = 2470 out_bytes = 138515 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.82505e-15 total_out_bandwidth_utilization = 1.60191e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1259, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2955, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2270 in_bytes = 130115 out_pkts = 2385 out_bytes = 299651 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:31:843.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:31:843.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:31:843.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692931843701 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:31:843.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:32:497.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:497.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692932497823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:497.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:498.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:498.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692932498218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:498.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:498.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:32:499.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:32:499.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:499.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692932499066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:499.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:663.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:664.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692932663901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:664.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:664.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:664.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692932664155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44585 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:664.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:664.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:32:664.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:32:664.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:664.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692932664678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:32:664.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:32:875.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:32:875.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:15:32:875.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:32:875.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692932875323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:32:875.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:32:875.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:32:876.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692932875862 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1698 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 149 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1932 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 466 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6558 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1874 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4275, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8516, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:32:876.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:32:876.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:32:876.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:32:876.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:32:877.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:32:877.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:32:877.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:15:32:877.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:877.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692932877690 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1699 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 150 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1932 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 466 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6558 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1874 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4275, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8518, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:32:878.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:32:878.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:878.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692932877728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:32:878.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:32:878.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:32:878.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:32:878.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692932877829 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1699 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 150 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1933 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 466 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6558 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1874 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 4276, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 8518, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:32:878.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:33:155.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:33:155.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692933155571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:33:155.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:33:156.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:33:156.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692933156348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40755 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:33:156.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:33:156.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:33:156.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:33:156.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:33:156.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692933156744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:33:156.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:641.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:641.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692934641208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:641.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:641.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:641.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692934641790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34519 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:641.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:642.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:34:642.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:34:642.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:642.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692934642236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:642.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:298.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692935298943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:299.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:299.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692935299112 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:299.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:299.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:299.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:299.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692935299499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:299.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:299.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692935299503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:299.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:299.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692935299564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:299.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:299.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692935299793 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:299.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:299.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:299.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692935299812 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:35:299.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:300.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:300.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692935300337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:300.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:300.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:300.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692935300374 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:35:300.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:35:300.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:35:345.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:345.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692935345512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:345.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:345.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:345.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692935345705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:345.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:35:345.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:35:346.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:35:346.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:35:346.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692935346066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:35:346.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:300.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:300.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692936300795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:301.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:301.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:301.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692936301920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:301.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:302.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:36:302.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:36:303.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:303.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692936302372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:303.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:303.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:303.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692936302380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:303.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:303.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:303.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692936302565 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:36:303.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:36:303.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:303.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692936302634 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:36:303.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:36:303.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:303.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692936303386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:303.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:303.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:303.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692936303427 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:36:303.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:36:303.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:36:353.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:353.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692936352887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:353.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:354.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:354.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692936354005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43567 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:354.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:354.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:36:354.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:36:354.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:354.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692936354306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:354.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:372.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:372.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692936372188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:372.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:373.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:373.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692936373319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:373.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:36:373.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:36:373.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:36:373.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:36:373.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692936373783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:36:373.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:270.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.41:8085::10.0.0.41:51792 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:37:335.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:335.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692937335423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:335.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:335.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:335.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692937335581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:335.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:335.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:37:335.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:37:335.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:336.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692937335966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:336.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:368.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:368.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692937368871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:368.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:369.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:369.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692937369350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:369.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:369.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:37:369.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:37:369.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:369.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692937369939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:369.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:369.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:370.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692937369944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:370.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:370.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:370.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692937370004 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:37:370.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:37:370.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:370.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692937370090 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:37:370.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:37:370.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:370.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692937370108 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:37:370.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:37:370.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:370.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692937370637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:370.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:370.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:370.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692937370677 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:37:370.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:37:370.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:37:419.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:420.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692937419946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:420.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:420.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:420.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692937420150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:420.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:37:420.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:37:420.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:37:420.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:37:420.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692937420571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:37:420.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:353.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:354.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692938353920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:354.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:354.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:354.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692938354464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:354.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:354.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:354.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:354.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:354.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692938354731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:354.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:491.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:491.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692938491675 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:491.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:492.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:492.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692938492355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:492.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:492.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:492.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:492.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:492.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938492729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:492.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:710.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:710.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938710354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:710.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:710.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:710.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692938710806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:710.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:711.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:711.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:711.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:711.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938711426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:711.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:711.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:711.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938711433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:711.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:712.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:712.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938711583 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:712.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:712.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:712.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938711629 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:15:38:712.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:712.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:712.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938712159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:712.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:712.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:712.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692938712191 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:38:712.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:38:712.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:15:38:758.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:758.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692938758514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:758.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:758.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:758.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692938758682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:758.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:758.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:38:759.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:38:759.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:38:759.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692938759221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:38:759.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:262.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:262.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692939262074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:262.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:262.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:262.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692939262786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46871 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:262.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:263.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:39:263.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:39:263.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:263.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692939263096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:263.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:686.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:686.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692939686328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:686.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:686.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:686.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692939686488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:686.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:39:686.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:39:686.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:39:686.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:39:686.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692939686883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:39:686.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog