2025-09-01 Mon 02:07:28:119.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:28:119.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:119.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692448119803 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:28:119.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:28:120.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:120.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692448120370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:120.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:120.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:120.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692448120412 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:28:120.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:28:120.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:28:132.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:132.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692448132006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:132.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:132.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:132.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692448132176 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:132.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:132.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:28:132.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:28:132.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:132.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692448132543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:132.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:166.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:166.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692448166540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:166.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:167.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:167.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692448167095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:167.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:167.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:28:167.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:28:167.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:167.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692448167540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:167.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:919.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:919.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692448919558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:919.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:920.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:920.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692448920105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:920.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:920.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:28:920.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:28:920.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:28:920.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692448920493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:28:920.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:229.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:229.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449229696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:229.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:229.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:230.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449229904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41547 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:230.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:230.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:29:230.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:29:230.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:230.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449230379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:230.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:930.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:930.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692449930471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:930.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:931.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:931.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:29:931.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:29:931.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449931436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33663 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:931.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:931.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:932.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449931846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:932.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:932.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:933.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449931852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:933.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:934.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:934.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449932054 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:29:934.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:29:934.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:934.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692449932401 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:29:934.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:29:935.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:29:935.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:935.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692449935016 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:935.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:935.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:935.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692449935218 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:29:935.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:29:983.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:983.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692449983340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:983.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:983.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:983.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692449983520 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:983.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:29:983.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:29:983.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:29:983.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:29:983.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692449983847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:29:983.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:30:956.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:30:956.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692450956039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:30:956.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:30:956.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:30:956.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692450956240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35069 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:30:956.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:30:956.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:30:956.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:30:956.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:30:956.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692450956583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:30:956.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:31:372.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:372.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692451372570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:31:372.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:31:372.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:372.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692451372814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:31:372.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:31:373.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:31:373.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:31:373.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:373.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692451373275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:31:373.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:31:831.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:831.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692451831196 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 3473 dequeues = 3473 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 3519 dequeues = 3519 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692422852725 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692451373286 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:07:31:831.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:31:831.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:831.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692451831314 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 693 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 789 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2669 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 763 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1743, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3473, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:31:831.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:31:839.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:839.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692451839504 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:07:31:839.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:07:31:839.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:839.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692451839551 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1048 in_bytes = 122576 out_tpkts = 1102 out_bytes = 61223 exception_packets = 43 exception_packets_allowed = 31 raw_vhost_stats= [ name = vhost0 in_pkts = 1048 in_bytes = 122576 out_pkts = 1102 out_bytes = 61223 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 1302 out_bandwidth_usage = 3078 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.80337e-15 total_out_bandwidth_utilization = 1.6198e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1255, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2988, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 950 in_bytes = 54839 out_pkts = 1051 out_bytes = 122702 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:07:31:839.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:31:839.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:31:839.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692451839576 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:07:31:839.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:07:32:380.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:380.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452380702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:380.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:380.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:381.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692452380972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34411 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:381.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:381.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:32:381.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:32:381.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:382.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452381455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:382.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:382.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:382.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452381461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:382.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:382.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:382.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452381563 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:382.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:382.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:32:382.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:382.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452381611 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:32:382.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:382.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:382.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452382124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:382.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:382.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:382.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452382178 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:382.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:435.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:435.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452435354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:435.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:435.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:435.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692452435914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37635 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:436.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:436.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:32:436.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:32:436.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:436.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692452436327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:436.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:628.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:628.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452628679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:628.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:629.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:629.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452629183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35557 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:629.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:629.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:32:629.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:32:629.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:629.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452629599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:629.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:629.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:629.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452629605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:629.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:629.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:629.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452629662 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:629.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:629.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:629.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452629748 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:629.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:629.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:629.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692452629766 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:32:629.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:630.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:630.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692452630409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:630.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:630.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:32:630.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:630.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692452630454 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:630.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:676.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:676.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692452676569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:676.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:677.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:677.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692452677169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:677.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:677.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:32:677.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:32:677.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:677.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692452677657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:677.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:852.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:07:32:853.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:07:32:853.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:32:853.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692452852950 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:853.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:853.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:32:853.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:07:32:853.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:07:32:854.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692452853746 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 700 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 793 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 190 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2685 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 767 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1753, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3501, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:32:854.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:32:854.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:07:32:854.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:07:32:854.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:32:854.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Idle 2025-09-01 Mon 02:07:32:854.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:854.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692452854085 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 701 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 793 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 190 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2685 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 767 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1753, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3503, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:32:854.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:32:854.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:854.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692452854112 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:32:854.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:32:854.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:07:32:854.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:854.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692452854216 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 701 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 794 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 190 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2685 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 767 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1754, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3503, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:32:854.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:32:972.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:972.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692452972728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:972.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:973.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:973.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692452973199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:973.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:32:973.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:32:973.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:32:973.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:32:973.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692452973590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:32:973.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:33:453.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:33:453.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692453453560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:33:453.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:33:454.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:33:454.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692453454085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:33:454.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:33:454.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:33:454.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:33:454.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:33:454.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692453454886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:33:454.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:33:694.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:33:694.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692453694790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:33:694.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:33:695.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:33:695.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692453695248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:33:695.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:33:695.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:33:695.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:33:695.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:33:695.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692453695919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:33:695.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:117.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:117.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692455117458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:117.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:118.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:118.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692455118048 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45171 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:118.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:118.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:35:118.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:35:118.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:118.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692455118546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:118.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:118.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:118.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692455118554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:118.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:118.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:118.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692455118630 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:35:118.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:35:118.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:118.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692455118718 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:35:118.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:35:118.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:118.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692455118736 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:35:118.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:35:119.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:119.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692455119278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:119.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:119.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:119.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:35:119.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692455119323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:35:119.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:35:168.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:168.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692455168428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:168.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:169.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:169.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692455169115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:169.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:169.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:35:169.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:35:169.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:169.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692455169843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:169.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:489.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:489.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692455489098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:489.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:489.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:489.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692455489315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:489.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:489.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:35:489.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:35:489.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:489.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692455489751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:489.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:604.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:604.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692455604105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:604.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:604.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:604.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692455604321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:604.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:35:604.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:35:604.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:35:604.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:35:604.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692455604774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:35:604.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:205.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:205.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692456205030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:205.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:205.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:205.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692456205244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35801 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:205.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:205.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:36:205.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:36:205.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:205.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692456205554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:205.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:573.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:573.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692456573746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:573.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:574.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:574.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692456574347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39209 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:574.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:574.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:36:574.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:36:574.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:575.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456574951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:575.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:575.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:575.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456574955 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:575.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:575.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:575.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456575022 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:36:575.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:36:575.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:575.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456575074 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:36:575.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:36:575.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:575.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456575641 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:575.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:575.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:575.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456575684 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:36:575.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:36:575.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:36:624.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:624.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692456624804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:624.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:624.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:624.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692456624975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:625.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:36:625.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:36:625.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:36:625.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:36:625.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692456625420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:36:625.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:37:678.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:37:678.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692457678525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:37:678.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:37:679.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:37:679.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692457679005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:37:679.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:37:679.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:37:679.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:37:679.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:37:679.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692457679473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:37:679.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:38:239.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:38:239.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692458239733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:38:239.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:38:240.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:38:240.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692458240324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:38:240.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:38:240.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:38:240.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:38:240.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:38:240.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692458240599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:38:240.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:551.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:551.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692459551574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:551.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:552.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:552.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459552210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:552.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:552.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:39:552.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:39:552.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:552.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692459552617 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:552.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:593.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:594.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459593873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:594.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:594.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:594.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459594043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:594.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:594.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:39:594.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:39:594.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:595.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:39:595.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459594415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:595.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:595.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:595.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459594421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:595.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:595.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:595.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459594491 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:39:595.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:595.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:595.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459594545 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:39:595.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:595.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:595.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459595080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:595.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:595.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:595.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459595120 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:39:595.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:646.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:646.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692459646281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:646.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:646.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:646.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692459646835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:646.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:647.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:39:647.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:39:647.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:647.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459647151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:647.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:816.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:817.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692459816928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:817.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:817.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:817.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692459817572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41315 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:817.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:817.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:39:818.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:39:818.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:818.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459818140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:818.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:818.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:818.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459818155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:818.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:818.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:818.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692459818216 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:39:818.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:818.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:818.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692459818336 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:39:818.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:818.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:818.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692459818361 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:39:818.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:819.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:819.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692459819044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:819.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:819.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:819.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692459819091 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:39:819.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:39:819.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:39:868.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:868.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692459868327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:868.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:868.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:868.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692459868849 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:868.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:39:869.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:39:869.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:39:869.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:39:869.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692459869516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:39:869.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:40:682.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:40:682.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692460682251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:40:682.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:40:682.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:40:682.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692460682479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35475 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:40:682.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:40:682.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:40:682.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:40:682.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:40:682.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692460682808 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:40:682.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:40:905.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:40:905.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692460904992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:40:905.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:40:905.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:40:905.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692460905637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46545 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:40:905.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:40:905.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:40:906.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:40:906.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:40:906.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692460906051 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:40:906.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:128.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:128.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692462128752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:128.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:129.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:129.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692462129287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:129.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:129.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:42:129.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:42:129.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:129.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692462129710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:129.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:129.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:129.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692462129724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:129.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:129.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:129.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692462129778 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:42:129.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:42:129.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:129.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692462129866 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:42:129.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:42:129.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:129.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692462129885 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:42:129.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:42:130.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:130.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692462130486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:130.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:130.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:130.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:42:130.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692462130529 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:42:130.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:42:182.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:182.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692462182655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:182.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:182.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:182.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692462182865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42977 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:182.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:183.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:42:183.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:42:183.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:183.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692462183341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:183.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:662.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:663.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692462662952 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:663.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:663.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:663.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692462663414 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44581 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:663.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:663.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:42:663.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:42:663.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:663.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692462663762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:663.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:904.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:904.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692462904281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:904.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:904.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:904.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692462904856 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:904.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:905.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:42:905.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:42:905.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:905.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692462905306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:905.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:146.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:146.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692463146481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:146.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:146.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:146.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463146652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43317 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:146.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:146.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:43:147.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:43:147.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:147.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692463147109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:147.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:476.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:476.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463476741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:476.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:477.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:477.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692463477451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46755 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:477.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:477.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:43:477.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:43:477.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:477.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463477917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:478.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:478.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:478.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463477929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:478.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:478.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:478.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463477991 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:43:478.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:43:478.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:478.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463478040 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:43:478.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:43:478.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:478.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463478607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:478.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:478.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:478.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692463478665 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:43:478.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:43:478.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:43:530.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:530.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692463530762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:530.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:530.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:530.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692463530911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33683 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:531.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:43:531.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:43:531.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:43:531.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:43:531.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692463531312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:43:531.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:44:539.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:44:539.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692464539545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:44:539.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:44:539.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:44:539.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692464539724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33029 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:44:539.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:44:540.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:44:540.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:44:540.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:44:540.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692464540211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:44:540.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:45:037.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:037.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692465037271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:037.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:45:037.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:037.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692465037890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38699 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:037.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:45:038.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:45:038.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:45:038.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:038.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692465038323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:038.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:502.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:502.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692466502373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:502.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:503.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:503.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692466502990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38519 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:503.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:503.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:46:503.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:46:503.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:503.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692466503400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:503.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:659.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:661.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466659897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:661.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:660.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:46:661.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:661.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:46:661.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466660339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46873 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:661.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:661.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:661.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466661846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:661.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:661.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:661.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466661850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:661.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:661.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:661.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466661905 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:46:661.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:661.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:662.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466661950 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:46:662.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:662.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:662.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692466662470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:662.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:662.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:662.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692466662679 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:46:662.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:662.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:46:713.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:713.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692466712947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:713.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:714.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:714.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692466713975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:714.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:714.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:46:714.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:46:714.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:714.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692466714678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:714.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:901.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:901.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692466901420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:901.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:901.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:901.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692466901859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38867 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:901.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:902.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:46:902.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:46:902.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:902.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466902385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:902.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:902.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:902.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466902392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:902.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:902.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:902.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466902473 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:46:902.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:902.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:902.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466902570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:46:902.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:902.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:902.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466902595 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:46:902.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:903.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:903.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:46:903.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692466903154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:903.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:903.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:903.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692466903192 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:46:903.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:46:949.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:949.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692466949289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:949.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:949.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:949.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692466949692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:949.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:46:949.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:46:949.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:46:950.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:46:950.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692466949993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:46:950.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:47:740.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:47:740.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692467740834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:47:740.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:47:741.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:47:741.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692467741353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42129 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:47:741.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:47:741.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:47:741.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:47:741.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:47:741.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692467741910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:47:741.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:47:967.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:47:967.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692467967178 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:47:967.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:47:967.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:47:967.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692467967360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38525 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:47:967.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:47:967.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:47:967.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:47:968.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:47:968.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692467967996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:47:968.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:070.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:070.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692469070508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:070.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:070.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:070.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692469070693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45059 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:070.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:071.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:49:071.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:49:071.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:071.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692469071259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:071.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:071.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:071.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692469071274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:071.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:071.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:071.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692469071432 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:49:071.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:49:071.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:071.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692469071739 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:49:071.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:49:071.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:071.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692469071768 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:49:071.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:49:072.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:072.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692469072316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:072.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:072.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:072.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692469072355 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:49:072.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:49:072.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:49:117.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:117.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692469117611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:117.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:117.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:117.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692469117834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39357 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:117.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:118.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:49:118.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:49:118.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:118.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692469118303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:118.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:686.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:686.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692469686060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:686.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:686.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:686.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692469686706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:686.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:49:687.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:49:687.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:49:687.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:49:687.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692469687185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:49:687.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:002.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:002.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692470002160 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:002.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:002.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:50:002.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:002.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:50:002.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692470002337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:002.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:002.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:002.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692470002812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:002.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:139.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:139.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692470139545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:139.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:140.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:140.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692470140011 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35519 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:140.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:140.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:50:140.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:50:140.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:141.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470140531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:141.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:141.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:141.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470140539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:141.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:141.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:141.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470140605 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:50:141.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:50:141.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:141.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470140653 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:50:141.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:50:141.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:141.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470141200 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:141.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:141.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:141.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470141240 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:50:141.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:50:141.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:50:162.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:162.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470162402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:162.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:162.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:162.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692470162549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41541 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:162.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:162.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:50:162.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:50:162.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:162.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692470162896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:162.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:191.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:191.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692470191307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:191.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:191.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:191.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692470191797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46753 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:191.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:50:192.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:50:192.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:50:192.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:50:192.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692470192079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:50:192.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:218.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:51:218.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692471218290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:51:218.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:218.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:51:218.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:51:218.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:51:218.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692471218505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44635 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:51:218.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:218.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:51:218.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692471218935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:51:218.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:981.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:51:982.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:51:982.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692471981077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:51:982.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:982.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:51:983.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692471981721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:51:983.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:983.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:51:983.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:51:983.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692471983072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:51:983.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:271.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:271.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692473271085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:271.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:271.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:271.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692473271691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37905 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:271.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:271.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:53:271.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:53:272.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:272.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473272001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:272.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:675.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:675.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473674970 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:675.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:53:675.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:53:675.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:675.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:675.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473675163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42491 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:675.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:675.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:675.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473675570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:675.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:675.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:675.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473675577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:675.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:675.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:675.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473675639 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:53:675.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:676.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:676.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473675888 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:53:676.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:676.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:676.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473675936 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:53:676.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:676.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:676.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473676566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:676.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:676.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:676.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473676601 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:53:676.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:676.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:53:719.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:719.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692473719361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:719.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:719.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:719.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692473719552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37417 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:719.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:719.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:53:719.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:53:719.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:719.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692473719968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:720.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:720.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:720.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692473719975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:720.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:720.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:720.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692473720080 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:53:720.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:720.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:720.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692473720141 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:53:720.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:720.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:720.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473720716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:720.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:720.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:720.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473720760 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:53:720.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:53:720.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:53:725.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:725.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692473725914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:725.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:726.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:726.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692473726360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:726.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:726.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:53:726.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:53:726.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:726.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473726754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:726.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:768.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:769.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692473768924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:769.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:769.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:769.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692473769478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:769.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:53:769.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:53:770.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:53:770.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:53:770.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692473770086 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:53:770.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:743.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:744.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692474743915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:54:744.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:744.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:744.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692474744520 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43221 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:54:744.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:744.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:54:744.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:54:744.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:744.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692474744958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:54:744.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:760.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:760.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692474760214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:54:760.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:760.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:760.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692474760654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45391 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:54:760.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:760.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:54:761.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:54:761.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:761.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692474761196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:54:761.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:799.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:799.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692475799164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:799.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:799.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:799.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692475799743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:799.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:800.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:55:800.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:55:800.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:800.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692475800146 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:800.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:800.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:800.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692475800157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:800.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:800.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:800.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692475800233 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:55:800.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:55:800.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:800.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692475800327 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:55:800.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:55:800.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:800.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692475800345 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:55:800.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:55:800.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:800.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692475800881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:800.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:800.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:801.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692475800918 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:55:801.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:55:801.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:55:850.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:850.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692475850124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:850.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:850.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:850.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:55:850.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:55:850.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692475850314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:850.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:55:850.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:850.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692475850721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:55:850.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:797.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:797.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692476797191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:797.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:797.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:797.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692476797388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:797.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:797.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:56:797.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:56:797.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:797.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692476797862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:797.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:813.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:813.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692476813886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:813.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:814.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:814.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692476814064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34913 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:814.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:814.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:56:814.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:56:814.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:814.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692476814490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:814.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:903.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:903.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692476903315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:903.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:903.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:903.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692476903791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:903.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:56:904.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:56:904.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:56:904.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:904.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692476904317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:56:904.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:124.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:125.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692477124168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:124.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:57:125.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:126.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:126.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692477124358 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44489 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:126.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:126.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:57:126.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:126.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692477126115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:126.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:126.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:126.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692477126120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:126.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:126.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:126.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692477126186 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:57:126.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:57:126.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:126.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692477126238 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:07:57:126.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:57:126.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:126.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692477126923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:126.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:07:57:126.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:127.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:127.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692477126965 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:57:127.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:57:180.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:180.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692477180124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:180.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:180.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:180.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692477180356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:180.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:57:180.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:57:180.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:57:180.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:57:180.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692477180737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:57:180.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:206.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:207.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692478206891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:207.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:207.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:207.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692478207091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34553 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:207.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:207.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:58:207.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:58:207.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:207.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692478207525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:207.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:704.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:704.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692478704617 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:704.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:704.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:704.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692478704810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46155 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:704.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:705.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:58:705.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:58:705.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:705.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692478705165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:705.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:241.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:241.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692480241777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:241.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:242.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:242.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692480242344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41495 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:242.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:242.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:00:242.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:00:242.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:242.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692480242740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:242.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:720.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:720.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692480720453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:720.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:720.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:720.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692480720951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43973 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:720.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:721.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:00:721.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:00:721.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:721.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692480721448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:721.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:721.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:721.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692480721455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:721.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:721.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:722.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692480721954 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:00:722.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:00:722.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:722.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692480722012 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:00:722.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:00:722.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:722.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692480722538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:722.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:722.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:722.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692480722571 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:00:722.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:00:722.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:00:772.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:773.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:00:773.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:00:773.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692480772765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:773.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:773.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:773.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692480772950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:773.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:00:773.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:00:773.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692480773299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:00:773.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:046.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:046.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692481046032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:046.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:046.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:046.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692481046223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45101 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:046.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:046.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:01:046.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:01:046.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:046.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692481046775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:046.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:046.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:046.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692481046788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:046.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:046.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:046.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692481046845 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:01:046.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:01:046.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:046.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692481046953 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:01:046.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:01:047.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:047.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692481046970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:01:047.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:01:047.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:047.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692481047617 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:047.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:047.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:047.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692481047666 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:01:047.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:01:047.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:01:094.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:094.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692481094815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:094.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:095.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:095.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692481095266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39455 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:095.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:095.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:01:095.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:01:095.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:095.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692481095826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:095.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:691.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:691.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692481691475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:691.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:691.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:691.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692481691668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:691.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:691.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:01:691.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:01:692.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:692.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692481691984 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:01:692.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:01:831.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:831.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692481831490 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 3792 dequeues = 3792 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 3841 dequeues = 3841 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692452854179 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692481691990 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:08:01:831.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:01:831.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:832.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692481831679 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 758 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 860 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 206 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2915 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 833 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1902, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3792, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:01:832.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:01:839.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:839.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692481839717 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:08:01:839.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:08:01:839.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:839.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692481839766 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1354580704 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0 five_min_avg = 0.09 fifteen_min_avg = 0.13 ] cpu_share = 1.40833 ] ] 2025-09-01 Mon 02:08:01:839.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:01:839.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:839.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692481839774 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.40833 used_sys_mem = 2854240503 one_min_cpuload = 0 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:08:01:839.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:08:01:839.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:840.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692481839801 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1125 in_bytes = 128018 out_tpkts = 1180 out_bytes = 65355 exception_packets = 46 exception_packets_allowed = 34 raw_vhost_stats= [ name = vhost0 in_pkts = 1125 in_bytes = 128018 out_pkts = 1180 out_bytes = 65355 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1025 in_bytes = 58845 out_pkts = 1128 out_bytes = 128144 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:08:01:840.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:01:840.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:01:840.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692481839828 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:08:01:840.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:08:02:104.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:104.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482103957 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:104.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:104.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:104.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692482104553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:104.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:104.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:02:104.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:02:105.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:105.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692482104959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:105.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:341.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:342.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:02:342.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692482341347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:343.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:343.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:343.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692482341926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:343.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:343.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:02:343.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:343.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482343356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:343.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:343.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:343.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482343365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:343.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:343.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:343.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482343429 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:02:343.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:02:343.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:343.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692482343712 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:02:343.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:02:343.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:343.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692482343731 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:02:343.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:02:344.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:344.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692482344423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:344.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:344.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:344.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692482344459 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:02:344.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:02:344.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:02:395.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:395.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692482395622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:395.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:396.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:396.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692482396285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:396.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:396.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:02:396.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:02:396.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:396.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692482396764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:02:396.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:02:854.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:08:02:854.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:08:02:854.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:02:854.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482854419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:02:854.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:02:854.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:02:854.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:08:02:854.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:08:02:855.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482854713 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 762 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 863 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 207 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2926 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 836 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1909, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3814, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:02:855.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:02:855.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:08:02:855.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:08:02:855.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:02:855.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Idle 2025-09-01 Mon 02:08:02:855.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:855.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482855099 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 763 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 75 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 863 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 207 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2926 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 836 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1909, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3816, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:02:855.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:02:855.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:855.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482855125 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:02:855.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:02:855.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:08:02:855.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:02:855.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692482855228 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 763 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 75 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 864 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 207 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2926 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 836 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1910, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 3816, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:02:855.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:03:332.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:333.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692483332911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:333.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:333.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:333.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692483333593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:333.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:333.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:03:334.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:03:334.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:334.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692483334389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:334.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:816.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:816.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692483816101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:816.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:816.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:816.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692483816766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:816.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:816.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:03:817.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:03:817.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:817.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692483817052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:817.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:941.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:941.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692483941116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:941.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:941.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:941.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692483941740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:941.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:03:942.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:03:942.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:03:942.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:03:942.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692483942171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:03:942.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:094.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:095.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692484094930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:095.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:095.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:095.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692484095105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:095.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:095.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:04:095.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:04:095.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:096.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:04:096.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484095542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:096.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:096.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:096.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484095548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:096.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:096.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:096.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484095610 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:04:096.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:04:096.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:096.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484095659 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:04:096.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:04:096.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:096.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484096160 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:096.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:096.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:096.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484096199 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:04:096.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:04:144.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:144.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692484144380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:144.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:144.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:144.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692484144920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:144.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:145.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:04:145.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:04:145.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:04:145.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692484145299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:04:145.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:05:045.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:05:045.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:05:045.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692485045458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:05:046.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:05:046.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:05:046.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692485045650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:05:046.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:05:046.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:05:046.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:05:046.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692485046145 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:05:046.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:05:350.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:05:350.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692485350610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:05:350.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:05:351.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:05:351.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:05:351.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:05:351.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692485351163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37323 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:05:351.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:05:351.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:05:351.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692485351690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:05:351.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:06:918.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:06:918.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692486918344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:06:918.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:06:918.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:06:918.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692486918867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:06:918.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:06:919.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:06:919.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:06:919.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:06:919.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692486919404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:06:919.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:542.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:543.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:07:543.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487542332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:543.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:543.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:543.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487542817 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:543.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:543.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:07:543.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:543.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692487543824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:543.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:543.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:543.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692487543829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:543.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:543.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:543.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692487543885 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:07:543.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:544.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:544.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487544002 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:07:544.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:544.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:544.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487544020 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:07:544.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:544.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:544.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:07:544.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692487544599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:544.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:544.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:544.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692487544650 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:07:544.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:589.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:589.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692487589814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:589.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:590.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:590.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692487590003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38309 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:590.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:590.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:07:590.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:07:590.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:590.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692487590331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:590.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:813.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:813.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692487813155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:813.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:813.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:813.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692487813622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:813.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:813.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:07:813.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:07:814.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:814.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487814017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:814.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:814.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:814.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487814026 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:814.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:814.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:814.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692487814123 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:07:814.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:814.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:814.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692487814195 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:07:814.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:814.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:814.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692487814706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:814.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:814.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:814.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692487814747 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:07:814.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:07:814.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:07:865.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:866.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692487865896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:866.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:866.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:866.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692487866503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:866.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:07:866.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:07:866.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:07:866.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:07:866.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692487866964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:07:867.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:634.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:634.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692488634489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:634.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:634.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:634.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692488634696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:634.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:634.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:08:634.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:08:635.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:635.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692488635020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:635.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:929.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:929.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692488929147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:929.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:929.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:929.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692488929628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42667 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:929.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:929.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:08:930.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:08:930.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:930.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692488930223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:930.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:987.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:987.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692488987853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:987.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:988.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:988.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692488988280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41729 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:988.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:988.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:08:988.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:08:988.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:988.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692488988720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:988.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:988.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:988.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692488988727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:988.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:988.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:989.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692488988798 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:08:989.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:08:989.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:989.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692488988889 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:08:989.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:08:989.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:989.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692488988907 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:08:989.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:08:989.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:989.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692488989500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:08:989.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:08:989.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:08:989.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:08:989.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692488989705 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:08:989.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:09:036.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:09:036.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692489036669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:09:036.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:09:037.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:09:037.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692489037002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36319 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:09:037.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:09:037.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:09:037.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:09:037.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:09:037.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692489037448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:09:037.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:045.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:045.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692490045630 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:045.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:045.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:045.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692490045830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36763 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:045.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:046.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:10:046.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:10:046.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:046.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692490046313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:046.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:489.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:489.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692490489241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:489.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:489.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:490.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692490489932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:490.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:490.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:10:490.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:10:490.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:490.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692490490280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:490.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:730.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:730.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692490730424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:730.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:731.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:731.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692490730980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:731.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:10:731.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:10:731.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:10:731.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:10:731.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692490731479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:10:731.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:023.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:023.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692491023536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:023.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:024.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:024.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692491024123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:024.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:024.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:11:024.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:11:024.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:024.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692491024566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:024.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:024.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:024.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692491024574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:024.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:024.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:024.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692491024658 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:11:024.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:11:024.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:024.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692491024716 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:11:024.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:11:025.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:025.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692491025258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:025.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:11:025.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:025.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:025.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692491025285 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:11:025.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:11:075.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:075.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692491075517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:075.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:075.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:075.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692491075696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36165 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:075.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:11:075.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:11:076.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:11:076.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:11:076.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692491076058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:11:076.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:116.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:116.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692492116495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:116.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:117.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:117.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692492117065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:117.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:117.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:12:117.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:12:117.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:117.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692492117666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:117.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:129.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:129.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692492129185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:129.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:129.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:129.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692492129730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35811 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:129.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:129.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:12:130.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:12:130.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:130.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692492130095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:130.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:13:930.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:13:930.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692493930281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:13:930.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:13:930.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:13:930.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692493930462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:13:930.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:13:930.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:13:930.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:13:930.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:13:930.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692493930785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:13:930.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:306.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:306.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692494306449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:306.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:306.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:306.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692494306646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:306.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:306.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:14:306.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:14:306.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494306982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:307.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:307.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494306989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:307.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:307.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494307097 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:14:307.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:307.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692494307198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:14:307.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:307.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692494307216 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:14:307.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:307.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692494307771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:307.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:307.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:307.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692494307832 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:14:307.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:307.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:14:360.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:360.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692494359987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:360.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:360.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:360.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494360390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:360.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:360.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:14:360.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:14:360.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:360.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692494360901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:360.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:367.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:367.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494367536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:367.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:367.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:367.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692494367802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37259 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:367.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:368.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:14:368.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:14:368.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:368.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692494368289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:368.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:368.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:368.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692494368293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:368.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:368.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:368.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692494368339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:14:368.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:368.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:368.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692494368384 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:14:368.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:368.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:368.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692494368892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:368.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:368.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:368.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692494368933 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:14:368.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:14:369.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:14:419.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:419.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494419233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:419.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:419.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:419.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494419409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45085 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:419.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:419.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:14:419.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:14:419.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:419.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692494419789 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:419.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:15:288.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:15:288.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692495288086 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:15:288.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:15:288.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:15:288.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692495288561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37575 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:15:288.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:15:288.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:15:289.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:15:289.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:15:289.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692495289068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:15:289.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:15:446.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:15:446.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692495445981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:15:446.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:15:446.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:15:446.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692495446585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37789 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:15:446.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:15:446.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:15:447.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:15:447.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:15:447.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692495447056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:15:447.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:295.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:295.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692496295353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:295.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:295.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:295.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692496295644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46537 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:295.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:296.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:16:296.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:16:296.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:296.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496296505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:296.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:296.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:296.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496296529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:296.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:297.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:298.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496296773 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:16:298.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:16:298.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:298.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496296937 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:16:299.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:16:299.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:299.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496296979 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:16:299.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:16:299.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:299.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496297872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:299.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:299.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:299.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692496297918 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:16:299.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:16:299.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:16:350.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:350.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692496349864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:350.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:350.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:350.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692496350080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:350.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:16:350.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:16:350.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:16:350.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:16:350.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692496350886 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:16:350.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:341.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:341.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692497341006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:341.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:341.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:341.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692497341458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:341.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:341.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:17:341.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:17:341.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:341.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692497341775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:341.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:413.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:413.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692497413237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:413.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:413.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:413.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692497413806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45899 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:413.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:414.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:17:414.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:17:414.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:414.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692497414233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:414.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:463.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:463.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692497463206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:463.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:463.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:463.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692497463793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40729 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:463.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:464.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:17:464.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:17:464.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:464.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692497464182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:464.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:585.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:54808 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:17:596.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:54822 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:17:855.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:855.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692497854975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:855.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:855.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:855.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692497855626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35141 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:855.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:855.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:17:855.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:17:855.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:855.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692497855967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:856.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:856.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:856.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692497855974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:856.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:856.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:856.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692497856052 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:17:856.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:17:856.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:856.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692497856105 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:17:856.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:17:856.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:856.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692497856782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:856.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:856.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:856.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692497856827 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:17:856.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:17:856.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:17:909.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:910.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692497909958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:910.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:910.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:910.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692497910159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:910.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:910.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:17:910.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:17:910.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:17:910.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692497910574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:17:910.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:18:927.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:927.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692498927673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:927.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:18:927.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:927.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692498927860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:927.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:18:928.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:18:928.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:18:928.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:928.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692498928315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:928.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:19:412.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:19:412.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692499411986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:19:412.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:19:412.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:19:412.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692499412447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40903 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:19:412.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:19:412.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:19:412.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:19:412.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:19:412.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692499412838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:19:412.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:20:854.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:20:854.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692500854538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:20:854.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:20:855.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:20:855.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692500855080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36263 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:20:855.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:20:855.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:20:855.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:20:855.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:20:855.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692500855630 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:20:855.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:244.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:244.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501244329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:244.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:244.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:244.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692501244907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34521 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:244.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:245.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:21:245.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:21:245.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:245.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501245318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:245.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:245.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:245.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501245323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:245.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:245.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:245.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501245380 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:21:245.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:245.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:245.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501245427 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:21:245.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:246.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:246.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:21:246.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692501246140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:246.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:246.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:246.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692501246185 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:21:246.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:299.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:299.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501299351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:299.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:299.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:300.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692501299943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:300.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:300.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:21:300.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:21:300.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:300.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692501300299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:300.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:698.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:698.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692501698394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:698.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:698.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:698.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692501698594 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:698.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:698.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:21:699.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:21:699.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:699.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692501699087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:699.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:699.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:699.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692501699094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:699.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:699.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:699.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692501699187 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:21:699.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:699.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:699.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692501699411 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:21:699.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:699.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:699.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692501699430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:21:699.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:699.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:700.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692501699990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:700.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:700.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:700.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692501700023 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:21:700.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:21:700.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:21:751.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:751.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692501751223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:751.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:751.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:751.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692501751418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46753 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:751.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:21:751.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:21:751.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:21:751.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:21:751.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692501751829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:21:751.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:22:209.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:22:209.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692502209543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:22:209.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:22:210.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:22:210.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692502210307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39447 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:22:210.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:22:210.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:22:210.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:22:210.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:22:210.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692502210648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:22:210.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:22:778.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:22:778.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692502778143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:22:778.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:22:778.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:22:778.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692502778363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:22:778.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:22:778.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:22:778.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:22:778.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:22:778.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692502778807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:22:778.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:589.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:589.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692503588987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:589.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:589.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:589.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692503589472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42515 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:589.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:589.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:23:589.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:23:589.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:589.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692503589951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:590.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:590.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:590.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692503589962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:590.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:590.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:590.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692503590037 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:23:590.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:23:590.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:590.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692503590194 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:23:590.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:23:590.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:590.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692503590222 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:23:590.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:23:591.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:591.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692503591049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:591.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:591.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:591.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:23:591.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692503591103 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:23:591.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:23:641.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:641.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692503641226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:641.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:641.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:641.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692503641653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42357 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:641.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:23:641.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:23:642.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:23:642.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:23:642.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692503642149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:23:642.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:064.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:064.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692504064813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:064.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:065.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:065.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692504065327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41343 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:065.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:065.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:24:065.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:24:065.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:065.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692504065909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:065.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:596.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:596.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692504596318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:596.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:597.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:597.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692504596974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:597.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:597.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:24:597.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:24:597.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:597.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692504597419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:597.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:650.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:651.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692504650949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:651.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:651.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:651.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692504651304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:651.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:651.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:24:651.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:24:651.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:651.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692504651766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:651.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:959.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:959.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692504959777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:959.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:960.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:960.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692504960288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:960.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:960.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:24:960.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:24:960.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:961.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692504960805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:961.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:961.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:961.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692504960813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:961.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:961.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:961.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692504960882 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:24:961.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:24:961.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:961.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692504960930 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:24:961.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:24:961.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:961.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692504961487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:24:961.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:962.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:962.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692504961998 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:24:962.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:24:962.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:25:010.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:010.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692505010172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:25:010.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:25:010.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:010.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692505010731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:25:010.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:25:011.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:25:011.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:25:011.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:011.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692505011149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:25:011.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:25:920.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:920.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692505920293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:25:920.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:25:920.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:920.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692505920472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41867 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:25:920.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:25:920.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:25:920.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:25:920.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:920.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692505920919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:25:920.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:26:523.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:26:523.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692506523603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:26:523.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:26:524.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:26:524.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692506524302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38589 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:26:524.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:26:524.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:26:524.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:26:524.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:26:524.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692506524798 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:26:524.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:918.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:918.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692507918055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:918.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:918.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:918.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692507918677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:918.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:918.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:27:919.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:27:919.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:919.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507919110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:919.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:919.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:919.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507919118 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:919.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:919.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:919.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507919193 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:27:919.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:27:919.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:919.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507919239 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:27:919.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:27:919.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:27:919.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:919.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507919805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:919.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:919.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:920.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507919843 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:27:920.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:27:965.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:965.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692507964985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:965.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:965.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:965.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507965403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:965.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:965.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:27:965.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:27:965.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:965.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692507965925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:965.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:991.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:991.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507991053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:991.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:991.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:991.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692507991475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:991.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:27:991.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:27:991.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:27:991.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:27:991.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692507991934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:27:991.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:611.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:611.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692508611904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:612.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:612.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:612.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692508612088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:612.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:612.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:28:612.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:28:612.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:612.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692508612457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:612.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:612.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:612.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692508612461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:612.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:612.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:612.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692508612519 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:28:612.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:28:612.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:612.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692508612748 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:28:612.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:28:612.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:612.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692508612768 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:28:612.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:28:613.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:613.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692508613293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:613.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:613.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:613.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692508613348 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:28:613.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:28:613.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:28:658.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:658.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692508658464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:658.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:658.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:658.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692508658626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:658.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:28:658.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:28:658.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:28:658.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:28:658.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692508658938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:28:658.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:29:001.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:29:001.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692509001105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:29:001.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:29:001.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:29:001.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692509001749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:29:001.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:29:002.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:29:002.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:29:002.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:29:002.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692509002280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:29:002.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:29:703.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:29:703.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692509703140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:29:703.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:29:703.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:29:703.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692509703670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41169 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:29:703.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:29:703.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:29:704.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:29:704.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:29:704.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692509704140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:29:704.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:628.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:629.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510628949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:629.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:629.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:629.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692510629171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:629.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:629.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:30:629.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:30:630.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:30:630.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510629649 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:630.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:630.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510629687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:630.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:630.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510629737 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:30:630.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:30:630.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510629814 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:30:630.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:30:630.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510629832 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:30:630.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:30:630.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510630375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:630.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:630.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:630.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692510630413 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:30:630.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:30:678.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:678.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692510678556 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:678.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:678.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:678.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692510678714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:678.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:678.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:30:679.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:30:679.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:679.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692510679075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:679.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:964.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:964.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692510964426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:964.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:964.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:964.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692510964923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:964.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:965.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:30:965.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:30:965.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:965.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692510965505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:965.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:702.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:702.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692511702319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:702.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:702.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:702.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692511702916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34271 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:702.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:703.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:31:703.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:31:703.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:703.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692511703250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:703.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:741.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:741.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692511741279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:741.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:741.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:741.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692511741459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:741.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:31:741.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:741.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:31:741.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:741.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692511741906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:741.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:772.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:772.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692511772078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:772.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:772.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:772.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692511772551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46479 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:772.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:772.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:31:772.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:31:772.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:773.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692511772987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:773.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:773.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:773.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692511773002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:773.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:773.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:773.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692511773078 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:31:773.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:31:773.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:773.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692511773124 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:31:773.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:31:773.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:773.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:31:773.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692511773718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:773.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:773.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:773.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692511773755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:31:773.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:31:823.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:824.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:31:824.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692511823945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:824.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:824.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:31:824.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:824.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692511824119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38331 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:824.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:824.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:824.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692511824562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:31:824.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:31:831.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:832.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692511831837 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 4119 dequeues = 4119 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 4171 dequeues = 4171 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692482855193 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692511824603 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:08:31:832.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:31:832.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:832.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692511831933 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 823 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 77 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 935 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3167 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 905 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2067, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4119, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:31:832.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:31:840.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:840.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692511839998 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:08:31:840.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:08:31:840.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:840.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692511840039 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1213 in_bytes = 144594 out_tpkts = 1272 out_bytes = 70843 raw_vhost_stats= [ name = vhost0 in_pkts = 1213 in_bytes = 144594 out_pkts = 1272 out_bytes = 70843 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.77084e-15 total_out_bandwidth_utilization = 1.59107e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1249, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2935, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1114 in_bytes = 64207 out_pkts = 1216 out_bytes = 144720 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:08:31:840.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:31:840.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:31:840.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692511840068 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:08:31:840.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:08:32:841.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:32:841.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692512841718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:32:841.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:32:842.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:32:842.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692512842346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41791 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:32:842.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:32:842.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:32:842.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:32:842.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:32:842.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692512842667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:32:842.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:32:855.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:08:32:855.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:08:32:855.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:32:856.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692512855339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:32:856.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:32:856.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:32:856.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692512855966 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 823 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 79 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 936 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3170 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 906 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2069, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4127, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:32:856.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:08:32:856.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:08:32:856.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:32:856.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:08:32:856.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:08:32:856.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:32:856.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Idle 2025-09-01 Mon 02:08:32:856.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:32:856.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692512856402 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 824 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 936 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3170 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 906 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2069, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4129, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:32:856.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:32:856.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:32:856.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692512856428 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:32:856.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:32:856.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:08:32:856.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:32:856.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692512856515 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 824 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 937 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3170 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 906 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2070, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4129, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:32:856.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:33:614.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:33:614.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692513614026 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:33:614.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:33:614.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:33:614.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692513614751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:33:614.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:33:615.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:33:615.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:33:615.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:33:615.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692513615359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:33:615.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:709.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:709.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692514709717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:709.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:710.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:710.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692514710260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:710.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:710.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:34:710.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:34:710.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:710.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692514710725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:710.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:710.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:710.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692514710734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:710.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:710.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:710.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692514710828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:34:710.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:34:710.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:710.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692514710880 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:34:710.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:34:711.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:711.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692514711399 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:711.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:711.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:711.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:34:711.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692514711437 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:34:711.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:34:762.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:762.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692514762636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:762.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:762.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:762.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692514762837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:762.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:763.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:34:763.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:34:763.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:763.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692514763223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:763.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:786.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:786.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692514786854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:786.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:787.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:787.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692514787449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34905 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:787.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:34:787.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:34:787.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:34:787.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:34:787.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692514787764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:34:787.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:762.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:762.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692515762327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:762.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:762.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:762.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692515762774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:762.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:763.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:35:763.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:35:763.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:763.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692515763254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:763.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:951.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:951.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692515951361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:951.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:951.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:951.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692515951834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:951.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:952.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:35:952.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:35:952.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:952.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692515952188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:952.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:952.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:952.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692515952193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:952.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:952.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:952.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692515952276 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:35:952.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:35:952.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:952.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692515952382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:35:952.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:35:952.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:952.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692515952399 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:35:952.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:35:953.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:953.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692515953052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:35:953.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:35:953.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:35:953.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692515953096 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:35:953.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:35:953.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:36:003.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:36:003.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692516003279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:36:003.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:36:003.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:36:003.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692516003685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41915 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:36:003.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:36:004.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:36:004.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:36:004.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:36:004.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692516004160 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:36:004.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:36:967.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:36:967.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692516967312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:36:967.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:36:967.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:36:967.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692516967755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:36:967.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:36:968.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:36:968.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:36:968.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:36:968.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692516968248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:36:968.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:251.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:251.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692517251448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:251.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:251.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:251.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692517251621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39007 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:251.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:251.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:37:251.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:37:252.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692517252027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:252.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:252.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692517252033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:252.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:252.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692517252093 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:37:252.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:37:252.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692517252335 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:37:252.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:37:252.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692517252354 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:37:252.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:37:252.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692517252879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:252.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:252.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:252.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692517252919 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:37:252.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:37:252.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:37:305.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:305.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692517305103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:305.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:305.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:305.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692517305702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43395 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:305.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:305.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:37:305.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:37:306.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:306.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692517305995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:306.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:815.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:815.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692517815393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:815.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:815.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:815.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692517815605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35191 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:815.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:37:815.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:37:816.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:37:816.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:37:816.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692517816064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:37:816.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:350.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:350.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692518350194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:350.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:350.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:350.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692518350374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34757 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:350.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:350.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:38:350.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:38:350.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:350.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692518350908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:350.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:804.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:804.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692518804404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:804.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:804.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:804.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692518804946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:804.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:805.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:38:805.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:38:805.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:805.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692518805457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:805.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:963.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:964.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692518963954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:964.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:964.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:964.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692518964404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35223 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:964.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:964.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:38:964.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:38:964.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:965.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692518964863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:965.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:965.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:965.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692518964872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:965.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:965.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:965.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692518964967 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:38:965.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:38:965.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:965.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692518965017 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:38:965.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:38:965.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:965.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692518965632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:38:965.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:38:965.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:38:965.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692518965675 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:38:965.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:38:965.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:39:013.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:39:013.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692519013830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:39:013.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:39:014.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:39:014.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692519014459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:39:014.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:39:014.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:39:014.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:39:014.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:39:014.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692519014903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:39:014.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:39:915.116 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:39:915.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692519915084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:39:915.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:39:915.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:39:915.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692519915610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:39:915.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:39:915.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:39:916.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:39:916.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:39:916.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692519916236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:39:916.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:40:475.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:40:475.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692520475091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:40:475.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:40:475.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:40:475.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692520475292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46475 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:40:475.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:40:475.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:40:475.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:40:475.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:40:475.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692520475672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:40:475.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:740.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:740.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692521740271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:740.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:740.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:740.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692521740453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39773 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:740.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:740.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:41:740.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:41:740.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:740.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692521740857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:740.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:740.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:740.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692521740864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:740.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:740.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:741.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692521740941 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:41:741.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:41:741.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:741.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692521740990 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:41:741.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:41:741.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:741.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692521741587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:741.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:741.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:741.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692521741626 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:41:741.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:41:741.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:41:790.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:790.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692521790737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:790.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:791.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:791.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692521791142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:791.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:791.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:41:791.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:41:791.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:791.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692521791543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:791.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:950.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:950.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692521950404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:950.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:950.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:950.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692521950888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:950.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:41:951.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:41:951.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:41:951.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:41:951.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692521951536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:41:951.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:691.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:692.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522691679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:692.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:692.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:42:692.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:42:692.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:692.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522691862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:692.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:692.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:692.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522692361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:692.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:945.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:945.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692522945598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:945.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:946.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:946.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692522946186 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37495 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:946.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:946.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:42:946.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:42:946.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:946.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692522946603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:946.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:946.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:946.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692522946609 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:946.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:946.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:946.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692522946696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:42:946.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:42:946.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:946.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522946802 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:42:946.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:42:946.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:946.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522946821 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:42:946.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:42:947.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:947.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522947365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:947.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:947.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:947.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:42:947.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692522947408 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:42:947.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:43:000.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:43:000.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692523000583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:43:000.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:43:001.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:43:001.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692523001161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33693 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:43:001.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:43:001.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:43:001.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:43:001.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:43:001.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692523001559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:43:001.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:43:964.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:43:964.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692523964721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:43:964.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:43:965.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:43:965.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692523965236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:43:965.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:43:965.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:43:965.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:43:965.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:43:965.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692523965760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:43:965.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:111.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:111.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692524111821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:111.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:112.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:112.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692524112389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45035 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:112.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:112.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:44:112.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:44:112.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:112.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692524112756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:112.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:112.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:112.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692524112764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:112.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:112.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:112.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692524112842 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:44:112.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:44:112.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:112.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692524112932 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:44:112.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:44:113.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:113.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692524112968 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:44:113.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:44:113.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:113.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692524113776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:113.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:113.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:44:113.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:114.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692524113830 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:44:114.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:44:158.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:158.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692524158910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:158.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:159.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:159.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692524159314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36945 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:159.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:159.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:44:159.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:44:159.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:159.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692524159779 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:159.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:654.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:654.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692524654495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:654.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:654.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:655.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692524654988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33145 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:655.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:655.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:44:655.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:44:655.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:655.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692524655398 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:655.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:194.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:195.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692525194915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:195.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:195.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:195.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692525195515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:195.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:195.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:45:195.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:45:195.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:196.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692525195965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:196.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:695.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:695.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692525695684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:695.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:696.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:696.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692525696241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37575 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:696.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:696.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:45:696.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:45:696.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:696.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692525696684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:696.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:696.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:696.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692525696690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:696.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:696.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:696.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692525696807 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:45:696.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:45:696.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:696.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692525696891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:45:696.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:45:697.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:697.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692525697458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:697.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:697.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:697.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692525697505 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:45:697.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:45:697.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:45:745.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:745.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692525745700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:745.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:746.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:746.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692525746167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:746.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:746.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:45:746.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:45:746.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:746.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692525746606 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:746.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:765.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:765.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692525765862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:765.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:766.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:766.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692525766247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:766.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:45:766.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:45:766.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:45:766.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:45:766.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692525766656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:45:766.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:781.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:782.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692526781804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:782.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:782.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:782.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692526781981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:782.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:782.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:46:782.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:46:782.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:782.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692526782362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:782.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:47:320.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:47:320.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692527320157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:47:320.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:47:320.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:47:320.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692527320775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:47:320.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:47:321.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:47:321.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:47:321.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:47:321.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692527321289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:47:321.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:831.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:833.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692528831579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:833.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:833.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:832.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:48:833.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692528832249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43697 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:833.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:833.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:48:833.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:833.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692528833444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:834.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:834.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:834.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692528833452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:834.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:834.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:834.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692528833511 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:48:834.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:48:834.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:834.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692528833574 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:48:834.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:48:834.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:834.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:48:834.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692528834403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:834.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:834.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:834.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692528834444 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:48:834.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:48:883.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:883.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692528883547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:883.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:883.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:883.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692528883719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:883.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:884.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:48:884.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:48:884.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:884.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692528884177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:884.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:924.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:924.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692528924528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:924.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:925.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:925.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692528925030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44555 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:925.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:48:925.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:48:925.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:48:925.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:48:925.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692528925484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:48:925.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:946.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:946.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692529946373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:946.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:49:947.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:947.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:947.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692529946575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42789 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:947.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:947.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:49:947.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:947.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692529947200 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:947.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:978.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:978.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692529978786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:978.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:979.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:979.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692529979408 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:979.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:979.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:49:979.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:49:979.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:979.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692529979828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:979.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:979.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:979.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692529979836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:979.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:979.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:979.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692529979915 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:49:979.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:49:980.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:980.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692529980043 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:49:980.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:49:980.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:980.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692529980065 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:49:980.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:49:980.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:980.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692529980619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:49:980.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:49:980.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:49:980.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692529980668 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:49:980.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:49:980.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:50:029.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:50:030.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692530029852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:50:030.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:50:030.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:50:030.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692530030036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:50:030.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:50:030.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:50:030.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:50:030.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:50:030.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692530030405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:50:030.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:50:975.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:50:975.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692530975602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:50:975.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:50:976.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:50:976.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692530976199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40755 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:50:976.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:50:976.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:50:976.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:50:976.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:50:976.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692530976522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:50:976.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:425.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:425.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692531425473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:425.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:425.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:425.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692531425670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:425.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:426.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:51:426.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:51:426.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:426.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692531426158 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:426.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:426.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:426.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692531426172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:426.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:426.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:426.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692531426247 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:51:426.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:51:426.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:426.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692531426353 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:51:426.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:51:426.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:426.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692531426370 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:51:426.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:51:427.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:51:427.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:427.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692531426945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:427.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:427.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:427.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692531427032 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:51:427.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:51:474.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:474.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692531474174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:474.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:474.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:474.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692531474640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34869 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:474.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:474.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:51:475.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:51:475.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:475.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692531475175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:475.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:891.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:891.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692531891376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:891.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:891.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:891.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692531891572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:891.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:51:892.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:51:892.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:51:892.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:51:892.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692531892206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:51:892.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:519.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:519.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692532519354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:519.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:519.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:520.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692532519956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:520.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:520.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:52:520.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:52:520.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:520.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692532520383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:520.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:812.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:812.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692532812681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:812.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:813.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:813.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692532813347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41037 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:813.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:813.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:52:813.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:52:813.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:813.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692532813812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:813.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:957.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:957.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692532957682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:957.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:957.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:957.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692532957853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35649 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:958.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:958.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:52:958.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:52:958.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:958.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692532958303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:958.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:958.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:958.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692532958310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:958.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:958.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:958.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692532958370 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:52:958.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:52:958.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:958.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692532958420 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:52:958.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:52:958.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:958.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692532958931 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:52:958.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:52:959.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:52:959.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692532958970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:52:959.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:52:959.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:53:004.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:53:004.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692533004344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:53:004.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:53:004.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:53:004.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692533004586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:53:004.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:53:004.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:53:004.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:53:004.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:53:005.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692533004975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:53:005.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:53:968.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:53:968.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692533968156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:53:968.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:53:968.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:53:968.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692533968759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:53:968.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:53:968.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:53:969.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:53:969.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:53:969.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692533969162 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:53:969.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:54:608.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:608.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692534608519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:54:608.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:54:609.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:609.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692534609060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:54:609.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:54:609.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:54:609.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:54:609.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:609.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692534609489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:54:609.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:55:960.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:960.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692535960383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:55:960.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:55:960.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:960.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692535960837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:55:960.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:55:961.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:55:961.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:55:961.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:961.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692535961298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:55:961.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:55:961.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:961.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692535961304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:55:961.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:55:961.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:961.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692535961378 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:55:961.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:55:961.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:961.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692535961442 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:55:961.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:55:962.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:962.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692535962355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:55:962.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:55:962.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:55:962.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:962.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692535962390 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:55:962.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:015.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:015.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692536015565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:015.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:016.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:016.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692536016047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:016.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:016.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:56:016.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:56:016.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:016.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692536016505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:016.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:111.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:111.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692536111307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:111.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:111.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:111.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692536111485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33707 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:111.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:111.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:56:111.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:56:111.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:111.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692536111774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:111.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:881.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692536881950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:882.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:882.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692536882119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:882.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:882.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:56:882.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:56:882.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692536882455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:882.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:882.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692536882464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:882.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:882.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692536882536 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:56:882.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:882.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692536882784 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:56:882.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:882.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:882.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692536882831 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:56:882.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:883.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:883.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692536883371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:883.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:883.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:883.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692536883400 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:56:883.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:883.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:56:929.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:929.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692536929674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:929.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:929.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:929.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692536929902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:929.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:56:930.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:56:930.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:56:930.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:930.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692536930361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:56:930.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:57:036.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:57:036.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692537036189 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:57:036.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:57:036.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:57:036.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692537036419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42857 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:57:036.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:57:039.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:57:039.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:57:039.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:57:039.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692537039610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:57:039.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:57:893.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:57:893.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692537893496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:57:893.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:57:894.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:57:894.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692537894117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:57:894.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:57:894.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:57:894.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:57:894.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:57:894.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692537894578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:57:894.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:785.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:785.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692538785607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:785.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:786.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:786.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692538786116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40025 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:786.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:786.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:58:786.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:58:786.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:786.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692538786731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:786.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:786.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:786.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692538786740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:786.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:786.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:786.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692538786800 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:58:786.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:58:786.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:786.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692538786910 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:58:786.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:58:786.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:786.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692538786929 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:08:58:786.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:58:787.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:787.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692538787576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:787.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:787.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:787.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692538787655 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:58:787.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:58:787.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:08:58:839.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:839.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692538839781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:839.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:840.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:840.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692538840232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34135 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:840.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:840.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:58:840.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:58:840.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:840.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692538840630 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:840.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:965.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:965.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692538965790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:965.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:966.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:966.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692538966454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40381 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:966.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:58:966.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:58:966.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:58:966.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:58:966.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692538966807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:58:966.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:821.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:821.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692539821797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:821.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:822.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:822.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692539822350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:822.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:822.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:59:822.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:59:822.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:822.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692539822854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:822.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:856.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:856.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692539856709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:856.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:857.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:857.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692539857227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:857.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:857.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:59:857.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:59:857.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:857.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692539857719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:857.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:035.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:035.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692540035896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:036.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:036.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:036.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692540036250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36145 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:036.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:036.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:00:036.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:00:036.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:037.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:00:037.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540036536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:037.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:037.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:037.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540036545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:037.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:037.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:037.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540036603 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:00:037.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:00:037.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:037.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540036656 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:00:037.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:00:037.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:037.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540037220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:037.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:037.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:037.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540037263 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:00:037.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:00:083.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:083.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692540083385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:083.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:083.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:083.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692540083569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:083.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:00:083.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:00:084.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:00:084.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:00:084.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692540084090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:00:084.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:047.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:047.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692541047193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:01:047.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:047.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:047.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692541047369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40467 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:01:047.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:047.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:01:047.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:01:047.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:047.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692541047783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:01:047.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:695.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:695.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:01:695.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:01:695.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692541695000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:01:695.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:695.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:695.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692541695229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:01:695.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:695.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:695.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692541695552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:01:695.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:01:832.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:832.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692541832103 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 4415 dequeues = 4415 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 4470 dequeues = 4470 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692512856479 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692541695666 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:09:01:832.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:01:832.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:832.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692541832254 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 881 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1002 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 240 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3397 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 971 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2216, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4415, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:01:832.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:01:840.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:840.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692541840257 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:09:01:840.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:09:01:840.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:840.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692541840306 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1362977504 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0 five_min_avg = 0.07 fifteen_min_avg = 0.125 ] cpu_share = 1.11667 ] ] 2025-09-01 Mon 02:09:01:840.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:01:840.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:840.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692541840312 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.11667 used_sys_mem = 2854240503 one_min_cpuload = 0 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:09:01:840.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:09:01:840.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:840.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692541840340 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1289 in_bytes = 149994 out_tpkts = 1348 out_bytes = 74863 exception_packets = 49 exception_packets_allowed = 37 raw_vhost_stats= [ name = vhost0 in_pkts = 1289 in_bytes = 149994 out_pkts = 1348 out_bytes = 74863 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1187 in_bytes = 68101 out_pkts = 1292 out_bytes = 150120 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:09:01:840.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:01:840.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:01:840.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692541840367 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:09:01:840.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:09:02:856.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:09:02:856.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:09:02:857.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:02:857.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692542856674 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:02:857.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:02:857.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:02:857.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:09:02:857.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:09:02:857.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692542856977 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 881 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1002 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 240 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3397 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 971 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2216, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4422, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:02:857.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:02:857.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:09:02:857.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:09:02:857.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:02:857.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Idle 2025-09-01 Mon 02:09:02:857.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:02:857.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692542857418 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 882 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 85 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1002 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 240 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3397 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 971 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2216, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4424, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:02:857.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:02:857.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:02:857.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692542857445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:02:857.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:02:857.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:09:02:857.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:02:857.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692542857560 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 882 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 85 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1003 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 240 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3397 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 971 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2217, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4424, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:02:857.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:03:153.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:154.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692543153956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:154.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:154.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:154.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692543154150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41849 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:154.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:154.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:03:154.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:03:154.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:154.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692543154725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:154.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:214.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:215.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543214961 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:215.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:215.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:215.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:03:215.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:03:216.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:03:216.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543215161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:216.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:216.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:216.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543215539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:216.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:216.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:216.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543215543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:216.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:216.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:216.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543215615 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:03:216.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:216.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:216.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543215684 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:03:216.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:216.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:216.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543216402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:216.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:216.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:216.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543216446 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:03:216.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:269.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:269.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543269618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:269.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:270.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:270.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692543270282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:270.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:270.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:03:270.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:03:270.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:270.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692543270543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:270.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:565.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:565.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692543565822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:565.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:566.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:566.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692543566212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44037 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:566.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:566.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:03:566.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:03:566.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:566.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692543566776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:566.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:566.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:566.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692543566788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:566.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:566.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:566.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692543566847 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:03:566.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:566.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:566.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692543566946 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:03:566.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:567.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:567.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692543566995 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:03:567.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:567.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:567.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692543567597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:567.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:567.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:567.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692543567637 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:03:567.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:03:567.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:03:612.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:612.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692543612764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:612.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:612.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:612.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692543612947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:612.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:03:613.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:03:613.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:03:613.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:03:613.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692543613377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:03:613.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:04:260.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:04:261.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:04:262.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692544260694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:04:262.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:04:262.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:04:262.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692544261202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46029 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:04:262.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:04:262.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:04:262.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:04:262.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692544262753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:04:262.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:04:540.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:04:540.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692544540575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:04:540.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:04:540.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:04:540.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692544540756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:04:540.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:04:541.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:04:541.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:04:541.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:04:541.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692544541151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:04:541.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:403.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:05:403.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692545403670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:404.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:404.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692545403844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45411 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:404.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:404.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:05:404.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692545404273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:404.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:404.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692545404294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:404.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:404.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692545404367 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:05:404.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:05:404.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692545404495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:05:404.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:05:404.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:404.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692545404541 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:05:404.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:05:405.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:405.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692545405325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:405.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:405.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:405.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692545405384 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:05:405.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:05:405.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:05:455.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:455.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692545455749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:455.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:456.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:456.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692545456348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40205 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:456.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:05:456.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:05:456.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:05:456.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:05:456.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692545456835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:05:456.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:116.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:117.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692546116928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:117.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:117.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:117.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692546117437 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36993 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:117.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:117.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:06:117.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:06:117.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:117.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692546117804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:117.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:401.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:402.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692546401955 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:402.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:402.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:402.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692546402368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:402.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:402.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:06:402.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:06:402.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:402.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692546402857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:402.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:647.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:647.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692546647290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:647.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:647.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:647.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692546647756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:647.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:06:648.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:06:648.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:06:648.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:06:648.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692546648336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:06:648.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:330.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:330.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692547330878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:331.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:331.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:331.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692547331481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:331.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:331.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:07:331.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:07:331.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:331.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692547331950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:331.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:332.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:332.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692547331957 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:332.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:332.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:332.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692547332011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:07:332.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:07:332.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:332.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692547332067 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:07:332.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:07:332.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:332.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692547332632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:332.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:332.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:332.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692547332678 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:07:332.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:07:332.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:07:381.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:382.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692547381944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:382.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:382.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:382.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692547382117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:382.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:07:382.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:07:382.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:07:382.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:07:382.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692547382507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:07:382.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:275.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:08:275.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692548275060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:08:275.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:275.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:08:275.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692548275664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35459 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:08:275.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:275.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:08:276.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:08:276.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:08:276.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692548276173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:08:276.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:381.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:08:381.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692548381677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:08:381.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:381.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:08:381.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692548381876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:08:381.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:382.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:08:382.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:08:382.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:08:382.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692548382287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:08:382.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:150.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:150.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692550149993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:150.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:150.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:150.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692550150535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39043 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:150.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:150.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:10:150.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:10:150.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:150.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550150843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:150.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:150.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:150.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550150859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:150.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:150.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:150.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550150940 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:10:150.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:151.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:151.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550150996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:10:151.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:151.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:151.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692550151538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:151.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:151.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:151.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:10:151.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692550151592 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:10:151.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:202.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:202.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692550202756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:202.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:203.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:203.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692550203273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:203.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:203.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:10:204.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:10:204.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:204.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550204057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:204.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:356.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:356.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692550356512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:356.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:356.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:356.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692550356976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42385 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:357.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:357.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:10:357.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:10:357.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:357.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692550357496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:357.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:357.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:357.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692550357501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:357.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:357.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:357.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692550357581 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:10:357.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:357.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:357.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692550357660 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:10:357.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:357.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:357.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692550357678 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:10:357.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:358.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:358.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550358239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:358.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:358.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:358.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:10:358.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692550358332 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:10:358.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:10:409.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:409.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692550409658 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:409.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:410.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:410.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692550410301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:410.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:410.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:10:410.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:10:410.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:410.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692550410639 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:410.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:506.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:506.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692550506425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:506.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:506.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:506.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692550506884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:506.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:10:507.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:10:507.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:10:507.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:10:507.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692550507353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:10:507.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:167.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:167.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692551167289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:167.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:167.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:167.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692551167555 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43421 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:167.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:167.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:11:167.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:11:168.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:168.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692551167997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:168.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:418.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:418.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692551418829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:418.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:419.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:419.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692551419445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33709 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:419.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:419.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:11:419.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:11:419.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:419.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692551419713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:419.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:056.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:056.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552056331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:056.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:056.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:056.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692552056507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34941 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:056.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:056.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:12:056.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:12:056.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:056.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552056958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:056.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:056.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:056.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552056962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:057.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:057.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:057.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552057045 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:12:057.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:12:057.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:057.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552057311 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:12:057.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:12:057.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:057.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552057330 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:12:057.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:12:057.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:057.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552057932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:058.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:058.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:058.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:12:058.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692552057974 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:12:058.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:12:104.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:104.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692552104167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:104.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:104.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:104.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692552104351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:104.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:12:104.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:12:104.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:12:104.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:12:104.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692552104733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:12:104.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:103.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:104.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692553103940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:104.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:104.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:104.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692553104127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:104.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:104.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:13:104.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:13:104.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:104.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692553104590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:104.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:148.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:148.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692553148159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:148.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:148.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:148.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692553148781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:148.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:149.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:13:149.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:13:149.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:149.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692553149247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:149.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:363.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:364.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692553363911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:364.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:364.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:364.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692553364489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43837 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:364.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:13:364.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:13:364.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:13:364.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:13:364.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692553364862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:13:364.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:179.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:179.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692554179524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:179.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:180.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:180.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692554180159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:180.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:180.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:14:180.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:14:180.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:180.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692554180517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:180.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:180.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:180.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692554180525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:180.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:180.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:180.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692554180600 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:14:180.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:14:180.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:180.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692554180667 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:14:180.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:14:181.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:181.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692554181201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:181.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:181.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:181.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692554181241 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:14:181.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:14:181.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:14:233.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:233.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692554233382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:233.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:233.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:233.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692554233807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42099 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:233.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:234.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:14:234.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:14:234.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:234.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692554234198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:234.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:922.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:922.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692554922734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:922.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:923.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:923.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692554923023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:923.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:14:923.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:14:923.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:14:923.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:14:923.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692554923431 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:14:923.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:15:305.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:15:305.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692555305388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:15:305.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:15:305.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:15:305.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692555305661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:15:305.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:15:305.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:15:306.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:15:306.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:15:306.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692555306033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:15:306.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:037.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:037.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692557037009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:037.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:037.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:037.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692557037625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:037.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:037.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:17:037.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:17:037.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:037.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692557037965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:038.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:038.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:038.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692557037972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:038.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:038.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:038.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692557038054 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:17:038.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:038.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:038.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692557038177 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:17:038.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:038.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:038.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692557038198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:17:038.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:038.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:038.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692557038754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:038.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:038.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:038.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:17:038.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692557038797 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:17:039.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:087.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:087.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692557086975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:087.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:087.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:087.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692557087531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45143 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:087.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:087.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:17:087.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:17:087.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:088.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692557087974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:088.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:106.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:106.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557106154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:106.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:106.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:106.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557106322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:106.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:106.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:17:106.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:17:106.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:106.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692557106660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:106.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:397.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:397.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692557397504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:397.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:398.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:398.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692557397986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:398.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:398.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:17:398.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:17:398.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:398.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557398434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:398.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:398.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:398.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557398439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:398.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:398.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:398.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557398494 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:17:398.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:398.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:398.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557398548 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:17:398.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:399.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:399.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692557399058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:399.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:399.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:399.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692557399096 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:17:399.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:17:399.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:17:448.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:448.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692557448371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:448.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:448.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:448.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692557448781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:448.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:449.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:17:449.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:17:449.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:449.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692557449361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:449.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:18:114.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:18:114.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692558114129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:18:114.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:18:114.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:18:114.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692558114321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40247 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:18:114.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:18:114.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:18:114.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:18:114.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:18:114.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692558114768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:18:114.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:18:416.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:45058 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:18:427.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:45072 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:18:466.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:18:466.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692558466513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:18:466.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:18:466.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:18:466.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692558466681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:18:466.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:18:466.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:18:467.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:18:467.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:18:467.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692558467066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:18:467.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:063.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:063.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692559063577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:063.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:063.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:063.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692559063754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35771 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:063.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:064.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:19:064.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:19:064.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:064.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692559064161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:064.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:064.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:064.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692559064167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:064.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:064.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:064.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692559064225 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:19:064.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:19:064.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:064.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692559064571 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:19:064.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:19:064.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:064.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692559064599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:19:064.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:19:065.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:065.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692559065170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:065.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:065.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:065.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692559065198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:19:065.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:19:065.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:19:112.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:112.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692559112378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:112.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:112.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:112.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692559112552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37723 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:112.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:19:112.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:19:112.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:19:112.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:19:112.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692559112889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:19:112.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:058.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:059.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692560058047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:059.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:059.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:059.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692560058357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:059.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:058.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:20:060.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:20:060.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:060.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692560060042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:060.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:076.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:076.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692560076892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:077.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:077.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:077.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692560077090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:077.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:077.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:20:077.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:20:077.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:077.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692560077539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:077.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:321.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:321.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692560321265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:321.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:321.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:321.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692560321685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:321.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:322.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:20:322.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:20:322.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:322.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692560322183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:322.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:958.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:958.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692560958819 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:958.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:959.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:959.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692560959375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:959.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:959.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:20:959.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:20:959.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:960.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:20:960.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692560959713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:960.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:960.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:960.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692560959719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:960.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:960.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:960.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692560959778 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:20:960.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:20:960.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:960.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692560959828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:20:960.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:20:960.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:960.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692560960374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:20:960.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:960.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:20:960.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692560960415 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:20:960.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:21:011.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:21:011.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692561011563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:21:011.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:21:011.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:21:011.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692561011753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:21:011.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:21:012.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:21:012.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:21:012.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:21:012.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692561012203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:21:012.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:21:912.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:21:912.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692561912378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:21:912.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:21:912.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:21:912.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692561912902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41407 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:21:912.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:21:913.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:21:913.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:21:913.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:21:913.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692561913380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:21:913.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:22:166.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:22:166.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692562166246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:22:166.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:22:166.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:22:166.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692562166535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36911 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:22:166.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:22:166.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:22:166.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:22:166.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:22:166.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692562166881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:22:166.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:23:911.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:23:911.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692563911575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:23:911.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:23:912.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:23:912.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692563912245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45999 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:23:912.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:23:912.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:23:912.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:23:912.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:23:912.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692563912764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:23:912.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:289.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:289.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692564289727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:289.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:290.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:290.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692564290195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34855 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:290.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:290.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:24:290.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:24:290.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:290.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692564290700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:290.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:290.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:290.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692564290708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:290.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:290.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:290.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692564290780 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:24:290.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:290.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:290.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692564290869 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:24:290.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:290.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:290.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692564290904 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:24:290.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:291.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:291.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564291527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:291.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:291.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:291.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564291569 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:24:291.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:291.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:24:337.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:337.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692564337739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:337.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:337.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:338.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692564337942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36889 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:338.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:338.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:24:338.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:24:338.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:338.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692564338363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:338.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:426.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:426.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692564426353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:426.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:426.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:426.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692564426835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:426.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:427.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:24:427.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:24:427.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:427.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564427340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:427.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:427.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:427.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564427348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:427.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:427.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:428.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564427412 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:24:428.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:428.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:428.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564427460 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:24:428.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:428.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:428.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564428045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:428.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:428.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:428.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692564428094 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:24:428.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:24:428.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:24:474.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:474.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692564474261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:474.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:474.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:474.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692564474888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40797 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:474.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:24:475.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:24:475.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:24:475.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:475.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692564475241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:24:475.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:25:346.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:348.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692565346527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:25:348.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:25:347.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:25:348.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:348.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692565347050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40421 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:25:348.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:25:348.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:25:348.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:348.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692565348774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:25:348.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:25:411.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:411.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692565411330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:25:411.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:25:411.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:411.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692565411883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46831 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:25:411.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:25:412.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:25:412.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:25:412.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:412.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692565412279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:25:412.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:163.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:163.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692566163069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:163.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:163.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:163.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692566163856 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:163.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:164.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:26:164.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:26:164.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:164.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692566164229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:164.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:164.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:164.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692566164240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:164.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:164.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:164.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692566164343 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:26:164.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:26:164.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:164.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692566164486 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:26:164.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:26:164.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:164.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692566164515 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:26:164.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:26:165.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:165.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692566165366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:165.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:165.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:165.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692566165428 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:26:165.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:26:165.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:26:211.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:211.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692566211539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:211.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:211.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:211.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692566211715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:211.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:26:212.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:26:212.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:26:212.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:212.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692566212189 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:26:212.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:211.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:211.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692567211350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:211.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:212.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:212.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692567212059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37275 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:212.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:212.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:27:212.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:27:212.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:212.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692567212823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:212.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:248.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:248.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692567248433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:248.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:248.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:248.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692567248921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44297 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:248.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:249.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:27:249.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:27:249.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:249.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692567249422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:249.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:292.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:293.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692567292917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:293.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:293.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:293.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692567293455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:293.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:293.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:27:293.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:27:293.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:293.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692567293890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:293.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:944.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:945.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692567944905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:945.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:945.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:945.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692567945066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45203 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:945.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:945.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:27:945.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:27:945.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:946.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567945479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:946.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:946.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:946.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567945484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:946.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:946.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:946.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567945578 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:27:946.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:27:946.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:946.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567945638 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:27:946.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:27:946.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:946.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567946173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:946.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:946.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:946.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567946209 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:27:946.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:27:946.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:27:991.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:991.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567991405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:991.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:991.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:991.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692567991593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37649 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:991.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:27:991.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:27:991.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:27:992.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:27:992.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692567992007 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:27:992.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:045.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:045.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692569045210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:045.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:045.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:045.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692569045751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43205 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:045.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:045.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:29:046.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:29:046.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:046.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692569046058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:046.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:318.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:319.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692569318960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:319.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:319.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:319.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692569319549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:319.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:319.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:29:319.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:29:319.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:320.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692569319976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:320.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:029.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:029.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692571029593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:029.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:030.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:030.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692571030124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44635 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:030.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:030.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:31:030.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:31:030.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:030.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571030543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:030.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:030.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:030.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571030550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:030.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:030.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:030.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571030619 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:31:030.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:030.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:030.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571030678 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:31:030.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:031.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:031.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692571031250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:031.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:031.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:031.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:31:031.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692571031295 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:31:031.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:079.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:079.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692571079469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:079.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:079.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:079.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692571079644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:079.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:079.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:31:080.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:31:080.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:080.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692571080182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:080.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:098.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:098.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571098193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:098.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:098.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:098.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692571098757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43919 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:098.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:099.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:31:099.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:31:099.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:099.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692571099143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:099.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:470.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:470.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571470038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:470.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:470.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:470.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692571470508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46407 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:470.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:470.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:31:470.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:31:470.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:471.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692571470934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:471.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:471.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:471.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692571470941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:471.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:471.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:471.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692571471004 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:31:471.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:471.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:471.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692571471413 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:31:471.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:471.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:471.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692571471448 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:31:471.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:472.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:472.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692571472000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:472.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:472.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:472.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:31:472.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692571472058 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:31:472.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:31:518.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:518.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571518171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:518.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:518.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:518.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692571518607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39907 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:518.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:518.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:31:518.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:31:518.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:519.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571518981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:31:519.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:31:832.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:832.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692571832413 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 4733 dequeues = 4733 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 4791 dequeues = 4791 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692542857501 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692571518986 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:09:31:832.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:31:832.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:832.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692571832541 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 946 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 87 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1074 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 258 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3640 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1040 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2375, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4733, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:31:832.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:31:840.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:840.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571840533 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:09:31:840.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:09:31:840.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:840.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571840576 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1381 in_bytes = 166783 out_tpkts = 1444 out_bytes = 80567 exception_packets = 51 exception_packets_allowed = 39 raw_vhost_stats= [ name = vhost0 in_pkts = 1381 in_bytes = 166783 out_pkts = 1444 out_bytes = 80567 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.84132e-15 total_out_bandwidth_utilization = 1.60354e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1262, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 2958, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1280 in_bytes = 73679 out_pkts = 1384 out_bytes = 166909 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:09:31:840.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:31:840.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:31:840.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692571840600 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:09:31:840.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:09:32:025.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:025.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692572025305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:32:025.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:32:025.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:025.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692572025755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37067 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:32:025.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:32:026.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:32:026.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:32:026.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:026.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692572026452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:32:026.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:32:491.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:491.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692572491130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:32:491.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:32:491.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:491.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692572491714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:32:491.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:32:491.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:32:492.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:32:492.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:492.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692572492241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:32:492.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:32:857.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:09:32:858.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:09:32:858.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:32:858.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692572857740 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:32:858.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:32:858.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:32:858.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:09:32:858.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:09:32:858.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692572858419 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 946 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 89 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1076 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 258 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3646 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1042 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2379, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4744, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:32:858.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:32:858.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:09:32:858.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:09:32:858.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:32:858.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Idle 2025-09-01 Mon 02:09:32:858.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:859.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692572858825 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 947 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1076 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 258 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3646 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1042 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2379, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4746, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:32:859.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:32:859.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:859.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692572858853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:32:859.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:32:859.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:09:32:859.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:32:859.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692572858946 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 947 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1077 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 258 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3646 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1042 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2380, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 4746, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:32:859.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:33:244.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:244.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692573244148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:244.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:244.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:244.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692573244729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38315 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:244.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:245.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:33:245.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:33:245.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:245.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573245164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:246.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:246.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:246.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573245171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:246.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:246.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:246.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573245249 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:33:246.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:33:246.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:246.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573245351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:33:246.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:33:246.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:246.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573245382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:33:246.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:33:246.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:246.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573245983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:246.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:246.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:246.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692573246023 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:33:246.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:33:246.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:33:298.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:298.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692573298212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:298.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:298.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:298.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692573298389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36905 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:298.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:298.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:33:298.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:33:298.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:298.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692573298787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:298.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:826.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:826.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692573826593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:826.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:827.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:827.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692573827309 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33405 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:827.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:33:827.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:33:827.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:33:827.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:33:827.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692573827739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:33:827.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:34:316.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:34:316.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692574315965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:34:316.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:34:316.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:34:316.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692574316163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43401 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:34:316.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:34:316.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:34:316.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:34:316.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:34:316.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692574316549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:34:316.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:34:454.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:34:454.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692574454427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:34:454.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:34:455.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:34:455.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692574455033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39783 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:34:455.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:34:455.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:34:455.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:34:455.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:34:455.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692574455307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:34:455.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:383.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:383.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692575383294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:383.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:383.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:383.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692575383805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:383.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:384.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:35:384.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:35:384.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:384.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692575384211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:384.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:384.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:384.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692575384220 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:384.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:384.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:384.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692575384319 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:35:384.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:35:384.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:384.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692575384372 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:35:384.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:35:384.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:384.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692575384902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:384.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:384.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:384.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:35:384.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692575384928 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:35:385.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:35:431.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:431.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692575431067 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:431.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:431.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:431.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692575431616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:431.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:35:431.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:35:431.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:35:431.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:35:431.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692575431904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:35:431.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:36:242.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:36:242.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692576242679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:36:242.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:36:243.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:36:243.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692576243262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:36:243.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:36:243.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:36:243.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:36:243.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:36:243.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692576243594 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:36:243.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:36:467.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:36:467.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692576467067 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:36:467.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:36:467.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:36:467.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692576467256 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:36:467.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:36:467.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:36:467.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:36:467.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:36:467.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692576467771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:36:467.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:37:967.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:968.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692577967907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:37:968.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:37:968.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:968.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692577968100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:37:968.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:37:968.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:37:968.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:37:968.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:968.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692577968619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:37:968.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:37:968.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:968.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692577968635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:37:968.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:37:968.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:968.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692577968687 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:37:968.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:37:968.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:968.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692577968735 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:37:968.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:37:969.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:969.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692577969252 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:37:969.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:37:969.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:37:969.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:37:969.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692577969276 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:37:969.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:38:020.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:020.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692578020453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:020.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:021.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:021.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578021020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:021.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:021.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:38:021.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:38:021.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:021.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692578021415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:021.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:285.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:286.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578285924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:286.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:286.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:286.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692578286116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:286.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:286.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:38:286.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:38:286.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:286.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692578286431 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:286.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:595.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:595.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578595536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:595.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:595.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:595.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578595767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:595.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:596.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:38:596.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:38:596.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:596.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578596281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:596.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:596.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:596.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578596294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:596.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:596.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:596.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578596391 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:38:596.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:38:596.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:596.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578596526 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:38:596.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:38:596.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:596.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578596553 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:38:596.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:38:597.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:597.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692578597353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:597.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:597.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:597.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:38:597.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692578597412 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:38:597.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:38:647.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:647.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692578647603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:647.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:648.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:648.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578648330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:648.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:38:648.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:38:648.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:38:648.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:38:648.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692578648873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:38:648.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:39:047.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:39:047.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692579047611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:39:047.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:39:048.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:39:048.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692579048207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45691 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:39:048.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:39:048.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:39:048.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:39:048.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:39:048.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692579048554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:39:048.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:39:612.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:39:612.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692579612014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:39:612.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:39:612.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:39:612.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692579612207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:39:612.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:39:612.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:39:612.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:39:612.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:39:612.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692579612655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:39:612.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:347.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:347.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692580347789 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:347.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:347.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:348.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692580347982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:348.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:348.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:40:348.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:40:348.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:348.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580348370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:348.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:348.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:348.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580348384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:348.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:348.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:348.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580348443 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:40:348.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:40:348.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:348.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580348524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:40:348.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:40:348.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:348.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580348543 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:40:348.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:40:349.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:349.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580349093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:349.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:349.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:349.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692580349128 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:40:349.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:40:349.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:40:397.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:397.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692580397646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:397.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:398.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:398.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692580398252 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:398.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:398.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:40:398.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:40:398.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:398.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692580398615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:398.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:046.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:046.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692581046703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:046.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:047.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:047.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692581047183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:047.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:047.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:41:047.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:41:047.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:047.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692581047803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:047.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:433.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:433.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692581433738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:435.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:434.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:41:435.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:435.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692581434389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:435.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:435.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:41:435.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:435.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692581435622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:435.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:592.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:592.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692581592795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:592.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:593.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:593.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692581593387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:593.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:41:593.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:41:593.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:41:593.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:41:593.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692581593770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:41:593.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:498.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:498.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692582498625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:498.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:499.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:499.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692582499308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:499.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:499.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:42:499.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:42:499.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:501.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:42:501.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692582499638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:501.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:501.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:501.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692582499665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:501.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:501.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:501.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692582499761 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:42:501.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:42:501.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:501.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692582499825 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:42:501.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:42:501.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:501.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692582501102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:501.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:501.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:501.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692582501201 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:42:501.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:42:553.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:553.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692582553341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:553.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:553.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:553.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692582553624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:553.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:42:554.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:42:554.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:42:554.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:42:554.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692582554163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:42:554.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:43:517.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:43:517.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692583517310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:43:517.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:43:517.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:43:517.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692583517488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38765 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:43:517.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:43:517.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:43:518.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:43:518.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:43:518.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692583518223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:43:518.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:43:577.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:43:577.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692583577759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:43:577.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:43:578.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:43:579.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692583578978 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:43:579.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:43:579.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:43:579.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:43:579.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:43:579.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692583579394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:43:579.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:684.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:684.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692584683987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:684.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:684.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:684.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692584684544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:684.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:684.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:44:684.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:44:685.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:685.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692584685024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:685.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:685.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:685.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692584685030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:685.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:685.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:685.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692584685099 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:44:685.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:44:685.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:685.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692584685169 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:44:685.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:44:685.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:685.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692584685903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:685.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:685.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:686.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692584685943 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:44:686.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:44:685.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:44:733.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:733.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692584733129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:733.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:733.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:733.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692584733336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44557 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:733.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:44:733.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:44:733.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:44:733.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:44:733.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692584733654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:44:733.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:409.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692585409951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:410.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:410.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692585410130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44871 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:410.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:410.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:45:410.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:45:410.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692585410654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:410.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:410.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692585410660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:410.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:410.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692585410713 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:45:410.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:45:410.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692585410822 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:45:410.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:45:410.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:410.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692585410863 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:45:410.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:45:411.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:411.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692585411495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:411.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:411.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:411.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:45:411.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692585411537 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:45:411.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:45:462.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:462.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692585462659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:462.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:463.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:463.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692585463099 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:463.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:463.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:45:463.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:45:463.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:463.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692585463572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:463.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:606.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:606.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692585606408 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:606.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:606.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:606.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692585606589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42323 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:606.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:606.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:45:607.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:45:607.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:607.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692585607053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:607.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:795.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:795.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692585795808 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:795.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:796.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:796.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692585796421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37139 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:796.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:45:796.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:45:796.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:45:796.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:45:796.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692585796729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:45:796.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:453.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:454.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:46:455.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692586453777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:455.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:455.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:455.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692586453981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:455.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:455.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:46:455.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:455.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692586455608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:455.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:647.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:647.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692587647554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:647.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:648.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:648.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692587648111 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44679 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:648.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:648.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:47:648.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:47:648.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:648.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692587648451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:648.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:648.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:648.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692587648459 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:648.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:648.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:648.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692587648523 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:47:648.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:47:649.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:649.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692587649026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:47:649.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:47:649.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:649.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692587649045 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:47:649.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:47:649.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:649.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692587649661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:649.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:649.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:649.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:47:649.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692587649693 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:47:649.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:47:695.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:695.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692587695894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:695.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:696.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:696.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692587696093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34945 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:696.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:696.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:47:696.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:47:696.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:696.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692587696511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:696.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:920.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:921.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692587920918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:921.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:921.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:921.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692587921092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:921.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:921.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:47:921.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:47:921.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:921.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692587921471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:921.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:48:255.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:48:255.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692588255730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:48:256.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:48:256.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:48:256.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692588255938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:48:256.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:48:256.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:48:256.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:48:256.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:48:256.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692588256309 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:48:256.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:48:740.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:48:740.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692588740613 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:48:740.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:48:740.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:48:740.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692588740802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45001 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:48:740.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:48:741.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:48:741.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:48:741.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:48:741.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692588741263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:48:741.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:855.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:49:855.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:855.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:49:855.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589855194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:855.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:855.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:855.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589855396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36169 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:855.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:855.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:855.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589855892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:855.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:855.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:855.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589855896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:855.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:855.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:855.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589855954 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:49:856.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:49:856.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:856.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589856005 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:49:856.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:49:856.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:856.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589856645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:856.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:856.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:856.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692589856694 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:49:856.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:49:856.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:49:908.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:908.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692589908011 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:908.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:908.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:908.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692589908211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36785 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:908.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:49:908.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:49:908.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:49:908.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:49:908.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692589908629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:49:908.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:50:847.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:50:847.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692590847391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:50:847.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:50:847.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:50:847.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692590847842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36899 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:50:847.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:50:848.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:50:848.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:50:848.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:50:848.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692590848325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:50:848.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:50:862.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:50:862.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692590862725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:50:862.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:50:863.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:50:863.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692590863031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:50:863.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:50:863.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:50:863.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:50:863.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:50:863.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692590863401 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:50:863.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:521.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:522.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591521676 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:522.112 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:51:522.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:522.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:522.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:51:523.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:51:522.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591521917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33903 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:523.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:523.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:523.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591522453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:523.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:523.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:523.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591522458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:523.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:523.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:523.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591522524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:51:523.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:51:523.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:523.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591522573 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:51:523.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:51:523.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:523.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591523129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:523.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:523.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:523.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591523211 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:51:523.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:51:574.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:574.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692591574342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:574.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:574.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:574.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591574805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:574.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:51:575.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:51:575.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:51:575.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:51:575.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692591575553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:51:575.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:502.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:502.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692592502699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:502.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:503.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:503.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692592503450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:503.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:503.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:52:503.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:52:503.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:503.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692592503932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:504.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:540.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:540.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692592540453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:540.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:541.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:541.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692592541068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40521 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:541.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:541.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:52:541.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:52:541.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:541.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692592541560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:541.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:541.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:541.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692592541568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:541.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:541.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:541.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692592541633 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:52:541.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:52:541.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:541.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692592541733 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:52:541.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:52:541.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:541.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692592541751 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:52:541.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:52:542.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:542.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692592542441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:542.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:542.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:542.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:52:542.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692592542485 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:52:542.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:52:592.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:592.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692592592708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:592.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:593.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:593.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692592593234 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43085 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:593.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:593.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:52:593.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:52:593.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:593.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692592593896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:593.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:987.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:987.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692592987548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:987.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:988.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:988.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692592988017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42755 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:988.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:52:988.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:52:988.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:52:988.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:52:988.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692592988467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:52:988.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:53:638.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:53:638.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692593638102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:53:638.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:53:638.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:53:638.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692593638559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:53:638.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:53:638.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:53:638.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:53:638.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:53:639.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692593638982 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:53:639.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:54:628.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:628.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692594628120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:54:628.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:54:628.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:628.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692594628646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44187 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:54:628.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:54:629.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:54:629.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:54:629.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:629.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692594629239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:54:629.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:024.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:024.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692595024519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:024.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:025.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:025.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692595025039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:025.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:025.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:55:025.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:55:025.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:025.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595025777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:025.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:025.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:025.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595025783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:025.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:025.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:025.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595025844 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:55:025.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:55:025.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:025.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595025929 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:55:025.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:55:025.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:025.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595025946 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:55:026.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:55:026.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:026.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692595026712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:026.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:026.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:026.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692595026766 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:55:026.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:55:027.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:55:072.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:073.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692595072932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:073.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:073.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:073.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692595073476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:073.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:073.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:55:073.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:55:074.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:074.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692595074031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:074.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:619.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:619.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595619162 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:619.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:619.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:619.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595619563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:619.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:55:619.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:55:619.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:55:619.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:619.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692595619865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:55:619.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:001.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:001.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692596001217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:001.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:001.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:001.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692596001778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38395 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:001.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:002.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:56:002.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:56:002.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:002.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692596002245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:002.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:732.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:732.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692596732650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:732.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:732.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:732.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692596732933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:732.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:733.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:56:733.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:56:733.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:733.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692596733404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:733.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:733.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:733.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692596733412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:733.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:733.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:733.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692596733465 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:56:733.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:56:733.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:733.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692596733513 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:56:733.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:56:734.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:734.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692596734080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:734.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:734.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:734.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692596734125 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:56:734.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:56:734.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:56:786.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:786.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692596786557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:786.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:787.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:787.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692596787102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39315 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:787.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:56:787.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:56:787.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:56:787.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:787.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692596787379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:56:787.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:57:768.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:57:768.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692597768603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:57:768.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:57:769.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:57:769.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692597769260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46707 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:57:769.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:57:769.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:57:769.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:57:769.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:57:769.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692597769551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:57:769.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:57:928.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:57:928.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692597928406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:57:928.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:57:928.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:57:928.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692597928608 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44683 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:57:928.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:57:928.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:57:929.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:57:929.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:57:929.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692597929047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:57:929.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:301.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:301.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692598301438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:301.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:302.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:302.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692598302030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:302.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:302.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:58:302.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:58:302.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:302.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692598302548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:302.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:302.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:302.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692598302558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:302.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:302.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:302.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692598302674 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:58:302.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:58:302.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:302.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692598302746 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:58:302.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:58:303.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:303.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692598303366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:303.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:58:303.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:303.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:303.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692598303411 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:58:303.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:58:354.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:354.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692598354498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:354.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:355.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:355.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692598355073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:355.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:58:355.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:58:355.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:58:355.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:58:355.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692598355535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:58:355.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:264.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:264.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692599264707 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:264.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:265.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:265.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692599265297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:265.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:265.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:59:265.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:59:265.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:265.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692599265706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:265.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:328.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:328.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692599328085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:328.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:328.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:328.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692599328571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39495 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:328.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:328.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:59:328.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:59:329.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692599328995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:329.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:329.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692599329002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:329.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:329.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692599329101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:59:329.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:59:329.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692599329206 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:59:329.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:59:329.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692599329224 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:09:59:329.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:59:329.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692599329789 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:329.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:329.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:329.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692599329837 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:59:329.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:59:329.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:09:59:376.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:376.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692599376186 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:376.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:376.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:376.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692599376740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:376.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:377.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:59:377.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:59:377.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:377.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692599377239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:377.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:713.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:713.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692599713684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:713.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:714.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:714.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692599714188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40149 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:714.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:59:714.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:59:714.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:59:714.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:59:714.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692599714496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:59:714.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:331.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:331.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692600331426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:331.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:331.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:331.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692600331614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39487 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:331.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:331.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:00:332.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:00:332.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:332.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692600332114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:332.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:01:119.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:120.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692601119879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:01:120.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:01:120.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:120.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692601120124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:01:120.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:01:120.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:01:120.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:01:120.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:120.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692601120752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:01:120.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:01:832.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:832.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692601832722 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 5032 dequeues = 5032 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 5093 dequeues = 5093 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692572858909 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692601120777 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:10:01:834.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:01:834.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:834.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692601832842 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1004 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1142 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 274 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3873 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1107 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2526, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5032, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:01:834.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:01:840.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:840.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692601840751 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:10:01:840.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:10:01:840.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:840.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692601840793 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1362977504 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0 five_min_avg = 0.06 fifteen_min_avg = 0.115 ] cpu_share = 1.125 ] ] 2025-09-01 Mon 02:10:01:840.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:01:840.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:840.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692601840799 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.125 used_sys_mem = 2854240503 one_min_cpuload = 0 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:10:01:840.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:10:01:840.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:841.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692601840827 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 1457 in_bytes = 172151 out_tpkts = 1521 out_bytes = 84645 exception_packets = 54 exception_packets_allowed = 42 raw_vhost_stats= [ name = vhost0 in_pkts = 1457 in_bytes = 172151 out_pkts = 1521 out_bytes = 84645 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1354 in_bytes = 77631 out_pkts = 1460 out_bytes = 172277 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:10:01:841.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:01:841.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:01:841.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692601840852 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:10:01:841.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:10:02:033.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:033.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602033256 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:033.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:033.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:033.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692602033840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:033.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:034.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:02:034.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:02:034.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:034.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692602034238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:034.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:034.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:034.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692602034255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:034.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:034.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:034.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692602034336 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:02:034.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:02:034.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:034.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692602034430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:02:034.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:02:034.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:034.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692602034449 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:02:034.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:02:035.113 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:035.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692602035072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:035.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:035.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:035.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:02:035.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692602035115 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:02:035.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:02:080.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:080.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692602080282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:080.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:080.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:080.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692602080844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:080.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:081.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:02:081.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:02:081.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:081.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602081143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:081.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:312.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:312.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692602312294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:312.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:312.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:312.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692602312861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:312.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:313.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:02:313.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:02:313.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:313.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692602313336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:02:313.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:02:858.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:10:02:859.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:10:02:859.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:02:859.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602859076 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:02:859.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:02:859.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:02:859.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:10:02:859.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:10:02:860.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602859737 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1008 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1145 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 275 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3884 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1110 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2533, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5054, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:02:860.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:02:860.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:10:02:860.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:10:02:860.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:02:860.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Idle 2025-09-01 Mon 02:10:02:860.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:860.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602860157 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1009 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 95 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1145 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 275 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3884 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1110 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2533, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5056, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:02:860.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:02:860.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:860.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602860184 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:02:860.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:02:860.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:10:02:860.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:02:860.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692602860272 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1009 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 95 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1146 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 275 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3884 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1110 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2534, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5056, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:02:860.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:03:026.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:026.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692603026342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:026.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:026.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:027.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692603026964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40553 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:027.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:027.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:03:027.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:03:027.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:027.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692603027325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:027.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:926.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:926.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692603926646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:926.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:927.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:927.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692603927126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:927.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:927.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:03:927.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:03:927.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:927.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692603927479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:927.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:927.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:927.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692603927486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:927.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:927.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:927.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692603927581 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:03:927.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:03:927.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:927.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692603927668 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:03:927.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:03:928.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:928.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692603928293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:928.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:928.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:928.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:03:928.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692603928347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:03:928.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:03:979.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:979.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692603979465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:979.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:979.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:979.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692603979927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34385 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:979.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:03:980.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:03:980.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:03:980.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:03:980.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692603980477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:03:980.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:917.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:917.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692604917563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:917.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:918.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:918.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692604918075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44801 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:918.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:918.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:04:918.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:04:918.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:918.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692604918474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:918.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:973.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:973.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692604972945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:973.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:973.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:973.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692604973322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40969 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:973.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:973.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:04:973.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:04:973.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:974.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692604973775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:974.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:974.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:974.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692604973792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:974.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:974.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:974.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692604973856 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:04:974.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:04:974.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:974.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692604973904 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:04:974.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:04:974.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:974.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692604974429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:974.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:974.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:974.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692604974475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:04:974.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:04:974.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:04:979.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:979.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692604979598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:979.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:979.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:979.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692604979777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40437 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:979.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:04:980.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:04:980.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:04:980.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:04:980.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692604980265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:04:980.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:027.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:027.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692605027745 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:027.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:028.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:028.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692605028366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:028.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:028.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:05:028.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:05:028.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:028.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692605028708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:028.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:913.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:913.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692605913406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:913.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:913.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:913.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692605913679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37147 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:913.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:913.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:05:914.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:05:914.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692605914051 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:914.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:914.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692605914058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:914.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:914.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692605914135 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:05:914.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:05:914.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692605914252 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:05:914.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:05:914.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692605914282 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:05:914.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:05:914.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692605914837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:914.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:914.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:914.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:05:914.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692605914879 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:05:915.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:05:967.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:967.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692605966977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:967.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:967.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:967.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692605967391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44137 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:967.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:05:967.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:05:967.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:05:967.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:05:967.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692605967808 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:05:967.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:081.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:081.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692606081877 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:081.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:082.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:082.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692606082317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45823 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:082.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:082.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:06:082.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:06:082.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:082.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692606082686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:082.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:816.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:816.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692606816421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:816.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:816.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:816.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692606816921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:816.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:817.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:06:817.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:06:817.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:817.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692606817403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:817.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:985.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:985.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:06:987.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692606985024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:987.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:987.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:987.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692606985229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:987.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:06:987.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:06:987.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:06:987.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692606987697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:06:987.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:134.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:135.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692608134892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:135.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:135.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:135.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692608135072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40927 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:135.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:135.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:08:135.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:08:135.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:135.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692608135490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:135.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:914.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:914.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692608914606 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:914.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:915.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:915.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692608915184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:915.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:915.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:08:915.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:08:915.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:915.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692608915486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:915.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:915.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:915.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692608915496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:915.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:915.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:915.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692608915579 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:08:915.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:08:915.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:915.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692608915669 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:08:915.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:08:915.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:915.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692608915690 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:08:915.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:08:916.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:916.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692608916344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:916.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:916.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:08:916.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:916.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692608916389 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:08:916.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:08:965.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:965.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692608965551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:965.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:965.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:965.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692608965776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:965.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:08:966.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:08:966.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:08:966.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:08:966.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692608966251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:08:966.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:09:057.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:057.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692609057829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:057.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:09:058.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:058.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692609058406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:058.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:09:058.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:09:058.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:09:058.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:058.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692609058784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:058.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:010.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:010.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692610010410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:010.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:011.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:011.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692610011183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43043 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:011.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:011.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:10:011.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:10:011.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:011.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692610011652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:011.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:741.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:741.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692610741596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:741.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:741.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:741.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692610741811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:741.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:742.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:10:742.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:10:742.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:742.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692610742264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:742.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:742.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:742.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692610742271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:742.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:742.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:742.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692610742372 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:10:742.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:10:742.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:742.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692610742446 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:10:742.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:10:743.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:743.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692610743153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:743.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:743.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:743.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692610743202 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:10:743.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:10:743.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:10:792.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:792.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692610792362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:792.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:792.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:792.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692610792561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44959 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:792.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:10:792.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:10:792.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:10:792.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:10:792.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692610792942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:10:792.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:11:756.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:11:756.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692611756110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:11:756.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:11:756.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:11:756.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692611756611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35741 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:11:756.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:11:756.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:11:757.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:11:757.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:11:757.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692611757185 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:11:757.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:027.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:028.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:12:031.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692612027863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:031.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:031.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:031.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692612028040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40483 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:031.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:031.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:12:031.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:031.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692612031880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:031.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:095.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:095.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692612095651 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:095.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:095.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:095.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692612095838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37479 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:095.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:096.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:12:096.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:12:096.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:096.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692612096193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:096.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:096.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:096.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692612096199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:096.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:096.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:096.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692612096278 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:12:096.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:12:096.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:096.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692612096323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:12:096.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:12:096.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:096.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692612096864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:096.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:096.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:096.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:12:096.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692612096908 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:12:097.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:12:149.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:149.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692612149064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:149.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:149.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:149.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692612149635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38499 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:149.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:12:149.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:12:150.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:12:150.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:12:150.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692612150101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:12:150.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:054.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:055.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692613054912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:055.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:055.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:055.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692613055475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35241 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:055.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:055.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:13:055.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:13:055.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692613055888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:056.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:056.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692613055895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:056.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:056.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692613055956 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:13:056.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:13:056.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692613056304 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:13:056.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:13:056.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692613056322 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:13:056.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:13:056.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692613056843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:056.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:056.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:056.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:13:056.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692613056931 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:13:057.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:13:059.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:059.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692613059166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:059.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:059.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:059.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692613059720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34309 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:059.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:060.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:13:060.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:13:060.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:060.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692613060167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:060.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:102.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:102.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692613102062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:102.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:102.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:102.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692613102740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:102.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:102.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:13:103.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:13:103.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:103.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692613103125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:103.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:737.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:737.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692613737353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:737.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:737.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:738.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692613737932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44209 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:738.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:738.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:13:738.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:13:738.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:13:738.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692613738365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:13:738.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:14:084.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:14:084.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692614084285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:14:084.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:14:084.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:14:084.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692614084739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34645 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:14:084.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:14:084.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:14:085.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:14:085.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:14:085.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692614085114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:14:085.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:076.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:076.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692615076260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:076.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:076.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:076.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692615076432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:076.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:076.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:15:076.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:15:076.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:076.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692615076896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:077.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:668.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:668.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692615668032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:668.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:668.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:668.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692615668612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:668.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:668.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:15:669.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:15:669.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:669.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692615669066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:669.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:669.114 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:669.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692615669075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:669.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:669.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:669.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692615669202 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:15:669.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:15:669.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:669.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692615669315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:15:669.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:15:669.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:669.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692615669333 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:15:669.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:15:670.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:670.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692615670057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:670.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:670.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:670.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692615670105 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:15:670.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:15:670.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:10:15:721.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:721.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692615721267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:721.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:721.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:721.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692615721683 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:721.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:721.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:15:722.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:15:722.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:722.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692615722113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:722.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:957.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:957.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692615957307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:957.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:957.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:957.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692615957780 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37769 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:957.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:957.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:15:958.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:15:958.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:958.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692615958093 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:958.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:16:757.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:16:757.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692616757303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:16:757.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:16:757.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:16:758.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692616757941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40627 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:16:758.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:16:758.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:16:758.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:16:758.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:16:758.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692616758379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:16:758.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:374.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:374.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692617374517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:374.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:375.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:375.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692617375126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39091 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:375.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:375.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:17:375.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:17:375.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:376.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692617375635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:376.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:376.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:376.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692617375643 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:376.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:376.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:376.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692617375706 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:17:376.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:17:376.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:376.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692617375764 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:10:17:376.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:17:376.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:376.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692617376372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:376.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:376.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:376.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692617376414 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:17:376.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:17:376.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871